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ABSTRACT 


An  average  case  analysis  of  the  Davis-Putnam 
procedure,  an  algorithm  for  testing  prepositional  satisfi- 
ability, is  presented.   A  variety  of  distributions  on  the 
set  of  problem  instances  is  assumed.   For  each  of  these 
distributions  a  polynomial  bound  is  obtained  on  the  expected 
time  complexity  of  the  algorithm.   When  the  uniform  distri- 
bution is  placed  on  the  set  of  problem  instances,  the  expected 

2 
time  of  the  Davis-Putnam  procedure  is  shown  to  be  0(rn  ), 

where  n  is  the  number  of  clauses  in  the  given  set  and  r  is 
the  number  of  distinct  atoms  in  the  set.   An  investigation 
of  the  relationship  of  the  Davis-Putnam  procedure  to 
resolution  based  procedures  extends  these  results  to 
resolution  and  to  important  restrictions  of  resolution  such 
as  the  t-linear  strategy.   The  power  of  the  Davis-Putnam 
procedure  is  related  to  that  of  resolution-type  proof  proce- 
dures, where  the  length  of  refutations  of  unsatisf iable  sets 
of  clauses  is  taken  as   a  measure  of  their  power.   The 
complexity  of  the  satisfiability  problem  is  examined  for  the 
special  case  of  Horn  sets.   Questions  related  to  transforming 
a  set  of  clauses  into  a  Horn  set  are  resolved. 


V 


I.   INTRODUCTION 

1.1   The  Satisfiability  Problem 

The  problem  of  testing  a  formula  of  the  propositional 
calculus  in  conjunctive  normal  form  to  determine  whether  or 
not  it  evaluates  to  true    for  some  truth-value  assignment 
to  the  variables  is  known  as  the  satisfiability   ppobZem . 
This  problem  plays  a  fundamental  role  in  many  areas  of 
Computer  Science,  especially   computational  complexity  and 
theorem  proving.   It  is  intimately  connected  with  a  major 
open  question  in  complexity  theory  —  to  determine  whether 
there  exist   algorithms  with  polynomial  time  complexity  for 
a  class  of  problems  known  as  the  WP-complete  problems. 
The  satisfiability  problem  (SAT)  was  the  first  of  many 
important  combinatorial  problems  shown  to  be  l\iP-complete 
[Cook,  1971;  Karp ,  1972].   These  problems  are  equivalent 
in  the  sense  that  the  existence  of  a  polynomial  time 
algorithm  for  any  one  of  these  problems  implies  that  they 
all  have  such  algorithms.   SAT,  because   it  has  been  the 
subject  of  extensive  research,  is  a  good  representative 
problem  from  which  to  study  this  class.   Logicians  have 
examined  the  propositional  calculus  as  a  formal  system  for 
100  years,  commencing  with  Frege's  Begriffsshrift   pi±)lished 
in  1879.   These  investigations  can  be  usefully  applied  to 
help  explore  questions  arising  in  complexity  theory.  One 
such  open  question  is  whether  M?    (the  class  of  problems 


computable   nondeterministically  in  polynomial  time)  is 
closed  under  complementation.   A  positive  answer  to  this 
question  can  be  shown  to  be  equivalent  to  the  existence  of 
a  proof  system  for  the  prepositional  calculus  in  which  the 
length  of  a  proof  of  a  formula  is  bounded  by  a  polynomial 
in  the  length  of  that  formula  ICook  and  Reckow,  1974]. 
This  is  a  natural  formulation  of  the  closure  question  that 
can  take  advantage  of  the  extensive  study  proof  systems 
have  undergone. 

SAT  is  important  not  only  as  an  avenue  for  studying 
NP-compieteness ,  but  also  because  of  its  role  as  a  founda- 
tion for  logistic  systems.   Any  decision  or  semidecision 
procedure  for  a  logical  theory  (based  on  classical  logic) 
must  incorporate  within  it  a  solution  to  SAT.   Thus,  lower 
bounds  on  the  complexity  of  SAT  apply  to  any  logical  theory. 
In  many  cases  this  embedding  appears  explicitly  as  part  of 
the  decision  or  semidecision  procedure.   When  this  occurs 
one  has  the  freedom  to  choose  the  best  known  algorithm  for 
SAT.   This  gives  a  practical  imperative  for  the  development 
of  effective  algorithms  for  SAT.   We  shall  demonstrate  that 
the  Davis-Putnam  Procedure  [Davis  and  Putnam,  1960;  Davis, 
Loveland  and  Logemann,  1962]   is  an  effective  algorithm  for 
SAT.   Decision  procedures  based  on  the  method   of  elimination 
of  quantifiers,  which  transform   a  formula  to  an  equivalent 
one  in  the  prepositional  calculus,  can  use  algorithms  which  are 
as   efficient  as  the  Davis-Putnam  Procedure.   Herbrand's  theorem 


is  the  basis  of  semidecision  procedures  for  first  order  logic 
in  which  instantiations  of  the  clauses  are  tested  for  preposi- 
tional satisfiability.   Some  theorem  provers  of  this  type, 
notably  the  linked-con junct  method  [Davis,  1963;  Yarmush,  1976] 
and  a  similar  one  described  by  Schwartz  [Schwartz,  1978] 
test   prepositional  satisfiability  explicitly. 

SAT  is  so  fundamental  that  diverse  problems  in  all  areas 
of  Computer  Science  embed  SAT  in  their  algorithmic  solutions. 
This  may  occur,  for  example,  in  the  processing  of  data  base 
queries  or  m  digital  circuit  design. 

The  question  of  whether  there  is  a  polynomial  time 
algorithm  for  testing  satisfiability  (i.e.  P  =  ?  MP)    is 
a  fundamental  open  question  of  complexity  theory.   The 
prevalent  view  is  that  ?  ^    N?      so  that  there  are  no  polynomial 
time  algorithms  for  any  of  the  WP-complete  problems.   The 
evidence  for  this  conjecture  is  the  failure  of  many  researchers 
to  find  a  polynomial  time  algorithm  for  an  WP-complete 
problem,  despite  the  attention  given  to  this  broad  class  of 
problems.   On  the  other  hand,  no  significant  lower  bounds 
have  been  proved  for  any  WP-complete  problem.   A  proof  that 
P  7^  WP   does  not  appear  imminent. 

From  the  perspective  of  complexity  theory,  SAT  is 
considered  to  be  intractable  by  presently  available  algorithms. 
Intractability  is  usually  equated  with  exponential  worst 
case  time  complexity.   Practitioners,  while  finding  these 
considerations  useful,  continue  to  attempt  to  solve  problems 


considered  intractable  whenever  the  advantages  of  obtaining 
a  solution  outweigh  the  computational  cost.   The  work  that 
has  been  done  in  mechanical  theorem  proving  has  these 
characteristics.   In  these  practical  contexts   average 
case  analysis,  based  upon  the  actual  distribution  of 
problem  instances  encountered,  is  of  greater  interest  than 
the  worst  case  analysis.   Unfortunately,  good  average  case 
results  are  difficult  to  obtain  because  properties  of  the 
actual  distribution  may  be  unknown,  and  the  technical 
problems  encountered  in  performing  such  an  analysis  may  be 
formidable.   Worst  case  analyses  give  upper  bounds  on  the 
expected  time  complexity  of  an  algorithm  and  so  conservatively 
estimate  the  algorithm's  actual  performance.   If  these 
conservative  bounds  guarantee  acceptable  performance,  or 
if  they  are  close  to  a  known  lower  bound,  then  these  bounds 
are  particularly  informative.   However,  an  algorithm  should 
not  be  rejected  on  the  basis  of  a  disappointing  worst  case 
analysis.   Under  these  circumstances,  an  average  case 
analysis  may  indicate  that  the  algorithm  will  almost  always 

meet  desired  performance  criteria.   The  quicksort  algorithm 

2 
lAho,  Hopcroft  and  Ullman,  197GJ  which  has  an  0(n  )  upper 

bound  on  its  worst  case  performance  is  considered  a  more 
effective  algorithm  than  many  0(n  log  n)  sorting  algorithms. 
Quicksort  is  0(n  log  n)  on  the  average,  assuming  each  permu- 
tation of  the  keys  is    equally  likely.   The  simplex  method 
for  solving  linear  programming  problems  also  exhibits  this 


phenomenon.   Its  worst  case  analysis  yields  an  exponential 
bound  [Klee  and  Minty,  19721,  yet  acceptable  performance 
has  been  observed  in  practice  [Karp,  1976J. 

It  appears  that  this  is  the  case  for  the  satisfiabi  I  ily 
problem  as  well.   Namely,  when  tho  Davis-Putnam  procedure 
is  used  to  solve  the  problem  in  a  practical  context,  the 
observed  behavior  of  the  algorithm  is  such  that  tho  problem 
is  tractable.   However,   with  some  dirriculty,  expononi i nl 1 y 
difficult  cases  can  be  constructed  [Tseitin,  19()H|.   In 
this  dissertation,  an  average  case  analysis  of  the  Davis- 
Putnam  procedure  is  presented  that  supports  the  tractability 
of  this  W/'-complete  problem.   A  Tamil  y  of  results  is 
obtained  that  give   polynomial  bounds  for  various  diiiLi  ilni- 
tions  on  the  problem  instances.   Tn  pari  icular,  if  I  lio 
uniform  distribution  is  placed  on  IIh'  s.uiiplo  space  of 
problem  instances,  thc^n  the  expected  time  complexity  ot. 
the  Davis-Pul  n.iiii  procedure  is  shown  lo  he  ()(rn  )  ,  where  r 
is  the  nuiiiljer  of  atom:;  ,i|i|i(mi  in<|  in  I  lie  (jivtMi  sel  ol  cl.iu.si's 
and  n  is  thie  numljer  of  clauses  in  the  set.   It  shall  also 
be  argued  that  these  favorable  results  are  due  to  an 
essential  feature  of  the  Davis-Pul  n. mi  pio(i'(hire  and  nol  to 
favorable  distributional  nssiim[)t- i  ons .   It  is  also  a  conseiiinMice 
of  these  results  that  instances  on  which  I  he  nlqoi  iihin  does 
take  exponential  time  are  very  rare. 


1.2   Complexity  Theory 

Problems,  like  SAT,  whose  output  is  either  yes     (the  set 
is  satisfiable)  or  no    (it  is  not)  are  considered,  in  the 
framework  of  complexity  theory,  to  be  language  accepting 
problems.   An  alphabet,  Z  ,     is  a  finite,  nonempty  set  of 
distinct  symbols.  l    ,  the  set  of  words      over  Z ,  consists  of 
all  finite  sequences  of  symbols  of  Z.   A  language ,       L, 
is  any  subset  of  Z* .      The  language  accepting  problem  for 
L    is  "given  w  6  E*,  is  w  e  L?".   If  we  take  E  to  be  the 
set  of  all  symbols  needed  to  specify  formulas  of  the  preposi- 
tional calculus  in  conjunctive  normal  form,  then  we  can  let 
L   be  the  set  of  all  satisfiable  formulas.   We  are  particularly 

interested  in  languages  that  are  recursive ,   i.e.  such  that 

* 
there  is  an  algorithm,  AL,  which  takes   a  string  w  g  E   and 

eventually  halts  indicating  w  e  L   or  w  ^  L.   There  are 
many  underlying  models  which  give  precise  meaning  to  the 
notion  of  an  algorithm.   We  shall  assume  deterministic  Turing 
machines  as  our  computational  model.   The  results  obtained 
in  this  thesis  apply  to  any  reasonable  known  model  of  determin- 
istic computation.   See  [Garey  and  Johnson,  1979]  for  details. 
We  shall  also  consider   nondeterministic  computations  and  use 
nondeterministic  Turing   machines  as  our  computational  model. 
For  we  I* ,    let  |w  |  denote  the  length  of  w.   Let 
e"  =  {w  G  z   I  |w|  =  n}.   Let  AL  be  an  algorithm  that  accepts 
L.       The  loorst    case    time    complexity    of  AL  is  a  function 
\0RST,Al('^)  defined  as 


T         (n)  =   max   Lnumber  of  steps  AL  performs  given  input  w] 

I  w  I  =n 

Here  we  are  measuring  the  complexity  as  a  function  of  the 

most  difficult  instance  of  a  given  size.   An  alternative  is 

to  consider  an  average  over  all  instances  of  a  given  size. 

This  requires  that  a  probability  distribution  function  P 

be  placed  on  L    ,  for  each  n.   This  function  should  reflect 

the  probability  that  a  particular  instance  appears  when 

solving  the  problem  in  some  given  context.   Given  such 

distribution  functions  P 

n 

T^,^^_,„^^  ^T  (n)  =   y   P  {wl  [number  of  steps  AL  performs  given 

I w I =n  input  wj 

(Without  ambiguity  the  subscript  on  the  distribution  function 
will  be  dropped.) 

A  problem  which  is  solved  by  an  algorithm  whose  worst 
case  time  complexity  is  bounded  by  a  polynomial  in  the  length 
of  the  input  is  considered  tractable.   If  no  such  algorithm 
exists  it  is  considered  intractable.  Thus,  tractability  is 
considered  equivalent  to  membership  in  the  class 
?  =    {L    I  there  is  an  algorithm,  AL,  accepting  L    such  that 
Tj,_j,„^  ^^  (n)  <  en   for  some  constants  c  and  k}. 
WP  =  {L\    there  is  a   nondeterministic  algorithm,  NAL, 
accepting  L    such  that   ^   r    nal^"^^  -  ^^      ^°^   some  constants 
c  and  k}.   It  is  understood  that  a   nondeterministic 
algorithm  accepts  L    if  there  is  some  accepting  computation 
sequence  accepting  w  just  in  case  w  G  L.   Furthermore,  the 
number  of  steps  taken  by  the  shortest  such  sequence  is  used 


to  measure  the  time  complexity  of  the  algorithm  on  w. 
Because  many  diverse  and  fundamental  problems  including  SAT  . 
are  in   WP,  the  question  of  whether  P =  ^P   is  basic.  In  1971, 
Cook  [Cook,  1971]  showed  that  SAT  was  the  "hardest"  problem 
in  MP.   That  is,  if  SAT  is  in  P  then  P  =  NP.      This  was 
proved  using  the  notions  of  reducibilities  and  completeness 
that  were  developed  in  recursion  theory.   If  L  and  L'    are 

two  languages  over  ^'  then  L  is  ^-reducible    to  L '  if  there 

*     * 
is  a  function  f:  Z   -^  ^  ,  computable  by  a  deterministic 

algorithm  in  polynomial  time  such  that 

w  e  L  iff  f(w)  G  L'  . 

Clearly,  if  i    is  p-reducible  to  i.  '  and  L  '  is  in  P  then  L  is 

in  P  as  well.  i      is  NP-oomplete    if  L    is  in  MP  and  for  any 

L'  in  NP ,    L'    is  p-reducible  to  L.      Cook  showed  that  SAT 

is  WP-complete.   This  having  been  established,  any  other 

problem  in  MP  can  be  shown  to  be  NP-complete  by  proving  that 

SAT  is  p-reducible  to  it.   Karp  [Karp,  1972J  showed  that  in 

fact  most  problems  known  at  the  time  to  be  in  MP  but  not 

known  to  be  in  P  are  NP-complete.   In  fact,  although  many   ^ 

problems  have  been  added  to  the  list  of  WP-complete  problems 

(we  shall  add  another) ,  few  have  been  added  to  the  three 

problems  in  MP   that  could  not  and  still  have  not  been  shown 

to  be  either  in  P  or  WP-complete.   (They  are  GRAPH  ISOMORPHISM, 

COMPOSITE  NUMBERS  and  LINEAR  PROGRAMMING  [Garey  and  Johnson, 

1979] ) . 


1 .  3   Propositional  Calculus 

There  are  many  formal  systems  that  are    properly  called 
the  propositional  calculus.   They  each  specify  a  formal 
language  in  which  formulas    are  inductively  built  from  a 
countable  set  of  atoms    or  propositional    variables 
(denoted  p,  q,  x, jX^,.-.)/  using  connectives    such  as  &  (and), 
V  (or),  -^    (implies),     (negation)  in  a  manner  which 
guarantees  that  the  formulas  are  well    formed.      An  interpre- 
tation,   valuation    or  assignment    to    the    propositional    variables 
is  a  function  from   the  atoms  to  the  set  of  truth  values, 
B  -    {True, False } .   The  truth   value    of  a   formula   with  respect 
to  an  interpretation  is  computed  inductively  on  the  structure 
of  the  formula.   Connectives  are  assigned  their  standard 

meanings  as  Boolean  functions  of  the  proper  arity.   Thus  &  is 

2 
interpreted  as  a  function  from  B   ->  B  whose  value  is  True 

iff  both  its  arguments  are  True.   When  meanings  have  been 

assigned  to  the  connectives  and  an  assignment  to  the 

propositional  variables  is  made,  the  truth  value  of  a  formula 

can  be  computed.   The  choice    of  connectives  varies  with 

the  formulation  of  the  logic.   However,  all  Boolean  functions 

should  be  realizable  by  a  formula  of  the  language.   From  the 

point  of  view  of  complexity  theory  the  choice  of  connectives 

has  an  importance  beyond  being  able  to  realize  all  functions. 

Some  languages  can  realize  Boolean  functions  with  shorter 

formulas  than  others.   This,  together  with  the  choice  of 

rules  of  inference   and  axioms,  affect   the  power  of  logistic 

systems  to  prove  tautologies  concisely. 


The  language  attracting  our  greatest  attention  will 

utilize  the  three  connectives   &,  V,  and    .   Any  formula 

of  this  language  is  equivalent  (i.e.  realizes  the  same 

Boolean  function;  as  a  formula  in  Qonjunative    normal    form 

(CNF) .   A  formula  is  in  CNF  if  it  is  a   conjunction, 

C,  &  C^  &  . . .  &  C   ,  in  which  each  conjunct    or  clause, 
i     z  n  ^ 

C.  ,  i  =  l,...,n,  is  a  disjunction  of  literals,    L .  ,  V  L.„ 
1  il     i2 

V  ...  V  L.    ,  where  each  L. .  is  an  atom  or  a  negation  of 
an  atom.   Many  proof  systems  operate  exclusively  on  CNF 
formulas.   Formulas  in  CNF  are  frequently  written  as  sets 
whose  members  are  the  formulas '  clauses  which  are  themselves 
written  as  sets  or  strings  of  literals.   This  representation 
is  unambiguous  with  respect  to  equivalence  of  formulas 
because  both  conjunction  and  disjunction  are  commutative 
and  associative  operations.   A  literal  is  -positive    if  it  is 
an  atom,  negative    otherwise.   The  complement   of  a  positive 
(negative)  literal,  L,  is  the  negative  (positive)  literal,  L, 
with  the   same  atom.   We  shall  assume  that  a  clause  does 
not  contain  both  a  literal  and  its  complement.  (A  formula 
obtained  by  dropping  such  a  clause  is  equivalent  to  the 
original  formula.)   A  formula  is  satis fiable    if  there  is 
an  interpretation  for  which  the  formula  is  true.   A  formula 
is  a  tautology    if  it  is  true  for  every  interpretation. 
The  satisfiability   problem    is  to  identify  CNF  formulas  that 
are  satisfiable. 

A  formula  which  is  a  disjunction  of  conjunctions  of 
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literals  is  in  disjunctive    normal    form    (DNF) .   The 
tautology    problem    is  to  identify  DNF  formulas  that  are 
tautologies.   The  tautology  problem  and  satisfiability 
problems  are  duals.   If  $  is  a  formula  in  DNF,  then  (|)  is 
a  tautology  iff  ^  (which  can  easily  be  put  in  CNF  by 
interchanging  &'s  and  V's  and  replacing   literals  by  their 
complements)  is  unsatisf iable.  Decision  procedures  which 
test  a  DNF  formula  for  being  a  tautology  are  affirmation 
procedures;  those  testing  a  CNF  formula  for  satisfiability 
are  refutation    procedures. 

The  Davis-Putnam  procedure  (DPP)  is  a  refutation 
procedure  to  decide  SAT.   Given  a  set  of  clauses 
S  =  {C  ,C  ,...,C  1   the  procedure  applies  one  of  three 
different  rules.   Each  application  of  a  rule  (a  step  of 
the  procedure)  reduces  the  problem  of  satisfiability  of  S 
to  the  satisfiability  of  one  or  possibly  two  derived  sets, 
each  set  containing  fewer  literals  than  the  original. 
The  pure    literal    rule   can  be  applied  if  there  is  a  literal 
L  which  appears  in  a  clause  of  S  while  its  complement,  L, 
does  not.   Let  S,  =  {C  |  C  g  S  and  L  ^  C}.  Then,  S^  is 
satisfiable  iff  S  is  satisfiable.   For,  let  M  be  an  inter- 
pretation that  satisfies  S.   Since  S,  c  s,  M  satisfies  S^ 
as  well.   Conversely,  let  M  satisfy   S, .   Since  neither 
L  nor  L  appear  in  S, ,  it  is  permitted  to  assume  that  M  assigns 
True  to  L.   Then  M  will  satisfy  S. 

If  there  is  a  clause  consisting  of  only  a  single 
literal,  L,  (a  unit    clause)       then  the  unit    clause    rule   may 
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be  applied.   Let  S   =  {C  -  {L}  |  C  e  S  and  L  ^  C}.  Again, 
S  is  satisfiable  iff  S,  is  satisfiable.   For,  any  interpre- 
tation which  satisfies  S  must  assign  True  to  L   and, 
therefore,  L  is  assigned  False.   Thus  for  any  clause  C  €  S, 
M  satisfies  C  -  {L}  and  so  satisfies  S^ .   Conversely, 
let  M  satisfy  S^  .   Since  L  and  L  do  not  appear  in  S, 
assume  M  assigns  True  to  L.   M  satisfies  S.   Finally,  the 
splitting    rule   is  applied  whenever  neither  of  the  two 
other  rules  are  applicable.   Choose  any  literal  L. 
Let   S   =  {C  -  {L}  I  C  e  s  and  L  ^  C}   and  S^--    1C-{L}1CGS 
and  L  ^  C}.   Then,  S  is  satisfiable  iff  S,  or  S^  is 
satisfiable.  For,  suppose  S  is  satisfied  by  an  interpreta- 
tion M.   Either  L  or  L  is  assigned  True  by  M.  Then  either 
S,  or  S„  is  satisfied  by  M.   Conversely,  assiime,  without 
loss  of  generality,  that  S,  is  satisfied  by  M.  Since  L  and  L 
do  not  appear  in  S, ,  we  can  assume  that  M  assigns  True  to  L 
and  so  satisfies  S.   The  DPP  applies  a  rule  at  each  step 
reducing  the  satisfiability  of  S   to  the  satisfiability  of 
S   and  possibly  S- •   It  then  recursively   applies  itself  to 
these  problems.   Since  at  each  step  the  desired  sets  contain 
fewer  distinct  literals  and  fewer  clauses,  the  process  will 
eventually  terminate.   If  at  any  stage  a  set  has  a  clause 
containing  no  literals,  i.e.,  the  empty    clause,    then  that  set 
is  unsatisfiable.   The  empty  set  of  clauses  is  vacuously 
satisfiable.   Because  the  unit  and  pure  literal  rules  reduce 
the  satisfiability  of  S  to  that  of  the  single  set  S-^    ,    they 
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are  the  more  desirable  rules  to  apply.   An  Algol-like 
description  of  the  DPP  follows. 

procedure    DPP(S); 

1.  •£/  S  =  jZ)  then    return    satisf iable;  end   if', 

2.  if  0  &    S   then   return   unsatisf iable;  end   if; 

3.  if    the  unit  rule  applies  to  S  then 

-z:/DPP(S  )  =  satisfiable 

then    return    satisfiable; 
else    return   unsatisf iable ; 
end    if; 
end   if; 

4.  if    the  pure  literal  rule  applies  to  S  then 

if   DPP(S  )  =  satisfiable 

then   return    satisfiable; 
else    return   unsatisf iable ; 

end    if; 

5.  else     (apply  the  splitting  rule)  ^/•  DPP  (S^)  ^satisf  iable 

or  DPP(S  )  -    satisfiable 

then    return    satisfiable; 
else   return    unsatisf iable; 

end    if; 
end    if; 
end   DPP; 
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1.4   Proof   Systems  for  the  Prepositional  Calculus 

Traditionally  a  proof  system  is  a  formalism  used  to 
help  identify  the  valid  formulas  of  a  logistic  theory. 
It  consists  of  a  set  of  axioms  and  rules  of  inference.  A 
proof   of   a  formula  cj)  is  a   sequence  of  formulas,  ending 
with  <t> ,    in  which  each  formula  of  the  sequence  is  an  axiom 
or  is  inferred  by  some  rule  of  inference  applied  to  formulas 
preceding   it  in  the  sequence.   If  a  formula  $  has  a  proof 
it  is  called  a  theorem   of  the  proof  system.   If  a  proof 
system  is  to  accomplish  its  task  of  identifying  valid  formulas 
it  must  be  sound,    that  is,  only  valid  formulas  should  be 
provable.   Furthermore,  it  must  be  possible  to  decide 
algorithmically  when  a  sequence  of  formulas  is  a  proof. 
In  fact  this  algorithmic  verification  should  be  computation- 
ally fast.   It  is  desirable,  but  not  always  attainable,  that 
a  proof  system  be  complete ,    that  is  that  every  valid  formula 
should  have  a  proof.   We  shall  adopt  a  definition  of  a  proof 
system,  due  to  Cook  [Cook  and  Reckow,  1974 j   which  emphasizes 
the  notion  of  proof  as  an  object  which  verifies  the  validity 
of  a  formula.   This  definition  will  capture   many  of  the 
important  properties  of  the  notion  of  a  proof  yet  admit 
as  proof  systems  a  broader  range  of  objects  than  those  based 

on  axioms  and  rules  of  inference.   A  proof  system    for  the 

* 
prepositional  calculus  is  a  function  F  from  a  subset  of  Z 

(Z  is  a  finite  alphabet,  not  necessarily  the   alphabet  of  the 

language  of  the  prepositional  calculus)  onto  the  set  of  valid 
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formulas  of  the  propositional  calculus,  which  can  be  computed 
in  polynomial  time.   If  F(w)  =   cj)  then  w  is  a  proof  of  (J). 
Since  the  range  of  F  is  only  valid  formulas,  F  as  a  proof 
system  is  sound.   F  is  complete  since  it  is  onto.   Since  F 
is  computed  in  polynomial  time,  the  verification  that  w  is 
a  proof  of  4)  is  computationally  efficient.  Refutation  proce- 
dures fall  into  this  framework  by  regarding  a  refutation 
of  (})  as  a  proof  of  cf) .   For  example,  the  DPP  may  be  regarded 
as  a  proof  system  where  the  proof  of  a  set  of  clauses  consists 
of  a  trace  of  the  computation  performed  by  the  algorithm 
when  given  that  set.   This  trace  may  be  the  tree  of  derived 
sets  generated  by  the  procedure.   Sufficient  detail  must  be 
given  so  that  verification  that  the  string  presented  is 
actually  the  trace  of  a  DPP  refutation  can  be  carried  out 
algorithmically  in  polynomial  time. 

A  proof  system,  F,  is  polynomially    bounded    if  there 
exists  a  polynomial  p  such  that  for  any  tautology  ^,    there 
is  a  proof  w  of  (f)  satisfying  iw|£  p  (  |  (j)  |  )  .   The  definition 
of  proof  system  allows  the  following  theorem  to  be  easily 
proved. 

Theorem  (Cook) .   MP  is  closed  under  complementation  iff 
there  exists  a  polynomially  bounded  proof  system  for  the 

propositional  calculus. 

Let  F,  and  F   be  two  proof  systems  for  the  same  formula- 
tion of  the  validity  problem  for  the  propositional  calculus 

(for  example,  they  both  attempt  to  decide  CNF  satisfiability). 
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F^  polynomially    simulates    (p-simulates)  F„  if  there  exists 
a  function  f  which  maps  a  proof  in  F„  of  4)  into  a  proof  in  F, 
in  polynomial  time.   Thus  proofs  in  F„  can  be  turned  into 
proofs  in  F,   with  only  a  polynomial  increase  in  the  length 
of  the  derivation.   Thus  if  F^  is  not  a  polynomially  bounded 
verification  system  neither  is  F_.   [Reckow,  1976  J  has 
compared  the  power  of  many  classical  proof  systems  using 
this  ordering.   Our  concern  will  be  focused  on  resolution 
related  proof  systems. 

Resolution  is  a  refutation  procedure  which  operates  on 
formulas  in  CNF.   Basic  to  the  procedure  is  the  out   or 
resolution   rule.   Given  2  clauses,  called  parent   clauses, 


I     I 


L  V  L   V  ...  V  Lj^  and  L  V  L^  V  L2  V  .  .  .  V  L^^ ,  ,  infer  the 
clause  L^  V  L2  V  ...  V  Lj^  V  l|  V  L2  V  ...  V  L^^ ,  as  an 
additional  clause  in  the  conjunction.   The  complementary 
pair  of  literals  L  and  L  are  called  the  annihilated 
literals.   The  derived  clause  is  called  the  resolvent . 
When  the  unit  clause  L  is  resolved  with  its  complement  L, 
the  empty  clause,  denoted  D,  is  the  resolvent.   Since  the   ^ 
.  empty  clause  is  unsatisf iable,   any  set  of  clauses  which  can 
'  derive  the  empty  clause  is  unsatisf iable.   Resolution  is 
sound  and  complete.   Given  a  set  of  clauses,  S, 
S  u  {resolvents  of  clauses  in  Si  is  equivalent  to  S. 
Furthermore,  if  S  is  unsatisf iable ,  then  the  empty  clause 

is  derivable  from  S.   The  resolution  rule  can  be  incorporated 

( 
I 

''    into  a  decision  procedure   testing  the  satisfiability  of  S. 
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repeat    forever; 

if  U  &    S    then     S  is  unsatisf iable. 

Find  a  pair  of  clauses  in  S  containing  complementary 
literals,  whose  resolvent  is  not  in  S .   If  no  such     v^^trO    (T 
pair  exists,  S  is  satisfiable.   Otherwise  add  the 
resolvent  to  S. 

end  repeat; 

The  length  of  the  resolution  proof  is  the  number  of  clauses 
generated  by  the  resolution  rule  necessary  for  the  deriva- 
tion of  the  empty  clause.   The  above  procedure  may  generate   ' 

i 
many  extraneous  resolvents.   There  are  various  ways  to 

display  the  proof  or  refutation.   One  way  is  to  enumerate 
a  sequence  of  clauses,  each  element  of  the  sequence  being 
either  a  clause  from  the  given  set  or  the  resolvent  of 
previous  clauses  in  the  sequence.   The  final  clause  of  the 
sequence  is  the  empty  clause.   Alternatively,  these  clauses 
can  be  labels  of  nodes  of  a  directed  acyclic  graph  (DAG) . 
In  such  a  graph  a  directed  edge  is  placed  from  clause  C,  to  C- 
iff  C„  is  obtained  by  resolution,  with  C-j^  being  one  of  its 
parents.   These  two  representations  are  nearly  identical; 
the  DAG  representation  explicitly   indicates  the  parents  of 
a  resolvent,  making  verification  of  the  correctness  easier. 
However,  since  recovering  the  parents  of  a  resolvent  is      ' 
computationally  easy,  the  difference  between  the  DAG  and 
linear  representation  is  not  essential.   In  a  third  varia- 

I 

tion,  derivations  are  represented  as  trees.   In  such  a 
representation,  a  clause  can  be  the  parent  of  at  most  one 
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derived  clause.   Thus,  if  a  clause  is  used  more  than  once 
in  a  derivation,  the  subtree  deriving  it  must  be  repeated. 
This  transformation  can  cause  an  exponential  blow-up  in 
the  number  of  nodes   needed  to  represent  a  derivation 
as  a  tree  instead  of  a  DAG.   Tseitin  showed  that  this  blow- 
up is  unavoidable  by  proving   that  tree  resolution  cannot 
p-simulate  resolution  proofs  presented  as  DAGs  [Tseitin, 
1968]  . 

Regular  resolution  is  a  restricted  form  of  resolution. 
A  resolution  derivation  represented  as  a  DAG  is  regular    if 
along  any  path  from  the  root  of  the  DAG  to  a  leaf  there  is 
no  atom  that  is  annihilated  twice.   Tseitin  [Tseitin,  1968] 
showed  that  regular  resolution  is  not  a  polynomially  bounded 
proof  system.   In  the  obvious  way,  the  regularity  condition 
can  be  imposed  on  tree  resolution  in  which  case  we  speak  of 
regular    tree    resolution .      For  tree  resolution,  the  regularity 
condition  is  easily  fulfilled.   Loveland  and  Tseitin  have 
each  shown  that  without  increasing  the  length  of  the  proof 
a  tree  resolution  proof  can  be  made  regular  [Tseitin,  1968, 
Loveland,  1970]  . 

There  are  many  other  resolution  based  proof  systems 
whose  origins  can  be  found  in  the  theorem  proving  literature 
of  the  late  1960 's  [Chang  and  Lee,  1973].  In  1965,  Robinson 
recognized  that  the  resolution  rule,  in  the  context  of  first 
order  logic,  is  particularly  amenable  to  machine  implementa- 
tion [Robinson,  1965]  .   There  is  a  simplicity  achieved  in 
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having  only  a  single  rule  of  inference.   More  important,  the 
number  of  possible  resolutions  that  can  be  performed  at  any 
stage  is   manageable,  usually  being  somewhat  less  than  ten 
[Schwartz,  1978],   Future  work  decreased  this  number  further   j 
by  specifying  conditions  that  must  be  met  before  a  resolution  I 
is  performed.   The  thrust  of  these  so-called  restriction 

y 

strategies  was  to  prune  the  search  space  of  possible  refuta- 
tions, yet  still  maintain  the  completeness  of  the  proof 
system.   Unfortunately,  many  of  these  strategies  were  apt 
to  prune  successful  as  well  as  unsuccessful  branches 
indiscriminately.   However,  some  of  the  strategies,  notably 
linear  resolution,  do  not  substantially  increase  the  lengths 
of  proofs  as  compared  with  those  obtainable  by  regular 
resolution  [Kowalski  and  Kuehner,  1971]  . 

Proof  systems  are  a  valuable  tool  for  the  investigation   ' 
of  the   WP  =  WP   question,  but  do  not  directly  aid  the 
development  of  good  algorithms  for  SAT.   Finding  a  powerful      l 
proof  system  is  equivalent  to  finding  a  good  nondeterrainistic 
algorithm  for  SAT.   At  each  step  of  its  computation  the 
nondeterministic  algorithm  guesses  the  correct  rule  of 
inference  to  apply.   A  good  algorithm  for  SAT  could  be  based 
on  the  proof  system  if  the    level  of   nondeterminism  is  not 
too  high.   Reckow's  study  of  proof  systems  [Reckow,  1976] 
puts  Frege   systems  (systems  like  P,  [Church,  1956J,  and 
others   familiar  from  logic  texts)  as  the  most  powerful, 
natural  deduction  systems  next,   followed  by  resolution  systems 
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and  truth  tables.   As  we  go  from  the  weakest   to  the  strong- 
est systems   the  nondeterminism  increases.   Thus,  the  choice 
of  a  proof  system  on  which  to  base  an  algorithm  for  SAT 
involves  a  tradeoff   between  length  of  proof  and 
nondeterminism. 


1.5   Review  of  Previous  Work  on  Expected  Time  Bounds 

The  unacceptable  worst  case  performance  of  an  algorithm 
can  be  the  result  of  a  few  rare  instances  on  which  the 
algorithm  performs  poorly.   Work  by  Karp  [Karp,  19  76] 
suggests  that  this,  in  fact,  may  be  the  case  for  many 
MP-complete  problems.   He   reports  results  in  which  poly- 
nomial time  algorithms  construct  near  optimal  solutions  on 
nearly  all  problem  instances.   For  example,  in  0(n  log  n)  time, 
almost  all  traveling   salesman  problems  can  be  solved  to 
within  1  +  e  of  an  optimal  solution.   Results  regarding 
approximation  algorithms  may  not  carry  over  to  exact 
algorithms,  but  do  encourage   the  search  for  exact  algorithms 
for  WP-complete  problems  having  good  average  case  performance. 

Among  the  first  algorithms  to  be  analyzed  for  average 
case  complexity  were  sorting  and  searching  algorithms. 
A  thorough  presentation  can  be  found  in  Knuth  [Knuth,  197  3] . 

Erdos  and  Renyi  introduce  the  noration  of  a  random 
graph  [Erdos  and  Renyi,  1959].  Using  this  model,  many 
graph  theoretic  algorithms  have  been  analyzed  [Karp,  1976 > 
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Grinunet  and  McDiarmid,  1975;  Spira,  1973;  Posa,  1976; 
Perl,  1977].   Complexity  results  have  been  so  favorable 
using  this  model   [Karp,  1976J,  that  the  relevance  of  the 
model  has  been  called  into  question. 

Recently,  diverse  combinatorial  problems  have  been 
the  subject  of  expected  time  analyses.   Yao  and  Yao 
[Yao  and  Yao,  1978J   looked  at  the  problem  of  selecting 
the  k    largest  key,  assuming  all  n!  orderings  equally  likely. 
[Doyle  and  Rivest,  1976;  Knuth  and  Schonhage ,  1977]  have 
analyzed  two  important  union-find  algorithms,  showing 
their  expected  linearity.  [Bentley  and  Shamos ,  1978]  give 
a  linear  expected  time  algorithm  to  find  the  convex  hull 
of  a  set  of  points  in  Euclidean  space.   They  emphasize  the 
"divide  and  conquer"  nature  of  the  algorithm  and  the  very 
weak  assumptions  made  on  the  distribution  function.  Further 
work  includes  [Floyd  and  Rivest,  1975  J  on  selection, 
[Guibas  and  Szemendi,  1976 J  on  hashing   and  [Yao,  1977] 
on  pattf^rn  rnatching. 
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1.6   Review  of  Previous  Work  on  the  Satisfiability  Problem 

Cook's  theorem  has  established  the  framework  from 
which  all  work  on  the  complexity  of  SAT  is  based.  In  a 
positive  direction,  the  search  continues  for  finding  better 
algorithms,  proof  procedures  and  approximation  schemes. 
In  a  paper  entitled  "Toward  Feasible  Solutions  to  the 
Tautology  Problem,"  Dunham  and  Wang  [Dunham  and  Wang,  1974J 
present  a  variety  of  algorithms  which  yield  efficient 
solutions  for  cases  which  defeat  the  well  known  algorithms. 
In  fact  it  is  difficult  to  find  a  class  of  instances  for 
which  none  of  the  methods  considered  will  work. 

A  popular  method  of  coping  with  WP-completeness  is 
to  consider  approximation  algorithms  which  give  near  optimal 
solutions  in  polynomial  time  (for  a  bibliography  see  [Garey 
and  Johnson,  1976]).   This  approach  is  useful  for  optimiza- 
tion problems,  like  the  traveling  salesman  problem,  in 
which  the  goal  is  to  maximize  an  objective  function.  For 
"yes/no"  problems  like  SAT  the  notion  is  not  as  useful. 
However,  there  exists  an  approximation  algorithm  for  the 
maximum  satisfiability  problem  —  find  the  assignment  which 
satisfies  the  maximum  number  of  clauses  in  the  set  [Johnson, 
1974].   The  algorithm  described  does  not  give  good  approxi- 
mations unless  the  number  of  literals  appearing  in  each 
clause  is  relatively  large.   [Lieberherr  and  Spector,  1978] 
prove  interesting  results  regarding  maximum  satisfiability, 
namely,  if  a  set  of  clauses  does  not  contain  a  pair  of 
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complementary  unit  clauses,  then  there  is  an  interpreta- 
tion obtainable  in  polynomial  time  which  satisfies  at 
least  hn  clauses,  where  h  is  the  golden  ratio 

(  (/5  -  l)/2  :;  0.618),  and  n  is  the  number  of  clauses. 
Furthermore,  h  is  maximal,  that  is,  for  any  larger  h  the 
problem  of  finding  such  an  interpretation  is  WP-complete. 

Negative  results  on  SAT  give  exponential  lower  bounds 
for  algorithms  based  on  particular  proof  procedures.  The 
most  important  result  of  this  type  is  due  to  Tseitin 

[Tseitin,  1968]  ,  who  proved  that  regular  resolution  is  an 
exponential  proof  system.   [Galil,  19  77]   improved  the 
result  slightly  by  increasing  the  lower  bound  on  the 
length  of  regular  resolution  proofs  from  0(2    )  to  0(2  ). 

[Kozen,  1977]  has  applied  the  same  idea  of  proving  lower 
bounds  for  natural  proof  systems  for  other  logical 
theories . 

Many  variations  on  SAT  have  been  examined.   The  3- 
satisf lability  problem,  in  which  clauses  are  restricted  in 
length  to  at  most  3  literals,  is  WP-complete  [Cook,  1971], 
whereas  2-satisf lability  is  polynomial.   For  interesting 
generalizations  and  problems  related  to  SAT  see  [Shaefer 
1978;  Garey,  Johnson   and  Stockmeyer,  1976;  Ladner,  1977]. 
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1.7   Summary  of  Results 

In  Chapter  II,  an  average  case  analysis  of  the  DPP 
is  presented.   Conditions  for  a  probability  distribution 
function  on  problem  instances  are  stated  which  assure 
that  the  expected  time  of  the  procedure  is  polynomial. 
The  variance  of  the  algorithm  is  discussed.   It  is  shown 
that  instances  on  which  the  DPP  requires  exponential  time 
are  rare. 

In  Chapter  III   results  are  presented  which  relate  the 
DPP  to  prior  work  on  proof  systems  for  the  prepositional 
calculus.   A  variant  of  the  DPP  (referred  to  as  DPP"), 
which  is  the  historical  predecessor  of  the  DPP,  is 
described.   DPP"  has  been  studied  in  this  context  because 
it  is  easily  regarded  as  a  particular  type  of  resolution 
procediire.   It  is  shown  that  a  structural  isomorphism 
exists  between  DPP  and  regular  tree  resolution.  Using  this 
characterization  of  the  DPP  proof  system  in  terms  of  resolu- 
tion, DPP  and  DPP"  are  compared.   It  is  found  that  they  are 
incomparable  with  respect  to  the  p-simulate  relation  on 
proof  systems.   Thus,  despite  surface  similarities,  fundamental 
differences  exist  between  DPP  and  DPP",   It  is  then  shown 
that  refutation  graphs  [Shostack,  1976]  can  be  quickly 
constructed  from   DPP  refutations.  From  a  refutation  graph 
other  types  of  resolution  proofs,  such  as  t-linear  refuta- 
tions, can  be  efficiently  obtained.  Thus,  the  results  of 
Chapter  II  extend  to  many  of  the  variants  of  resolution 
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discussed  in  the  theorem  proving  literature. 

In  Chapter  IV,  Horn  sets  and  their  relation  to  the 
DPP  and  resolution  is  discussed.   The  DPP,  modified  for 
this  special  case,  can  decide  satisfiability  in  linear 
time.   Without  modification  the  DPP  is  efficient  but  not 
linear.   It  is  shown  that  the  resolution  refutation  that 
can  be  generated  from  the  DPP  using  the  results  of 
Chapter  III   is  an  input  refutation.   It  is  the  shortest 
possible  resolution  proof,  consisting  of  only  as  many 
resolutions  as  there  are  distinct  atoms  in  the  set. 
The  DPP"  generates  unit  refutations  for  Horn  sets.  These 
refutations  are  generally  longer  than  the  input  refuta- 
tions but  are  still  linear  in  the  size  of  the  set. 

The  final  section  of  the  chapter  discusses  renamings, 
a  satisfiability  preserving  transformation  which  can 
sometimes  transform  a  set  into  a  Horn  set.   It  is  shown 
that  the  question  of  whether  a  set  is  renaraable  as  a  Horn 
set  is  decidable  in  polynomial  time,  but  the  problem  of 
maximizing,  by  renaming,  a  suitably  defined  notion  of 
degree  of  "Horn-ness"  is  WP-complete. 

These  results  can  be  taken  as  strong  evidence  that 
despite  the  NP-completeness  of  SAT,  the  problem  is,  in 
practice,  tractable.  Results  like  these  should  be  sought  for 
other  WP-complete  problems. 
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II.  AVERAGE  CASE  ANALYSIS  OF  THE  DAVIS-PUTNAM  PROCEDURE 

"Divide  and  conquer"  [Aho,  Hopcroft  and  Ullman,  1974] 
has  proved  to  be  a  useful  technique  in  the  design  of 
efficient  algorithms.   A  "divide  and  conquer"  algorithm, 
AL,  solves  a  problem,  S,  of  size  n,  by  dividing  S  into 
smaller  problems;  it  recursively  applies  AL  to  the  smaller 
problems  and  uses  the  partial  solutions  to  construct  the 
solution  of  S.   If  the  time  needed  to  divide  the  problem  is 
small  (for  example,  linear)  and  significantly,  if  the 
number  and  size  of  these  subproblems  is  also  small,  then 
an  efficient  algorithm  will  result.   The  DPP  is  a  "divide 
and  conquer"  algorithm  which  in  linear  time  divides  a  set 
of  clauses  into  at  most  two  smaller  sets.   However,  these 
sets  in  some  cases   will  not  be  of  substantially  smaller 
size  than  the  original.   As  a  result,  the  DPP  has  an  expon- 
ential worst  case  time  complexity.   Nevertheless,  as  will 
be  seen,  on  the  average,  the  subproblems  are  small  enough 
to  assure  that  the  expected   time  complexity  of  the  DPP  is 
polynomial . 

The  usual  modus  operandi   of  complexity  analysis  is  to 
parameterize   instances  of  a  problem  (see  Chapter  I)  by  the 
length  of  the  description  of  the  problem  presented  to  the 
algorithm.   The  complexity  of  the  algorithm  is  measured  as 
a  function  of  the  length  of  this  description.   For  example. 
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the  complexity  of  an  algorithm  to  test  primality  of  posi- 
tive integers  is  measured  as  a  function  of  the  number  of 
digits  of  the  number.   The  length  of  the  description  of 
a  problem  instance  is  a  crude,  problem  independent  measure 
of  the  difficulty  of  the  instance   relative  to  the  other 
problem  instances.   Better  results  can  frequently  be 
obtained  by  more  appropriate  parameterizations .  The  complex- 
ity of  graph  algorithms  is  better  described  as  a  function 
of  the  number  of  nodes  and  edges  in  the  graph.  In  a  similar 
fashion,  we  shall  parameterize  instances  of  the  satisfi- 
ability problem  using  two  parameters:   r,  the  number  of 
atoms  in  the  atom  set  from  which  the  clauses  are  chosen, 
and  n,  the  number  of  clauses  in  the  set. 

Let  C(r)  be  the  set  of  all  clauses  that  can  be 

constructed  from  the  atom  set    A  =  {x, ,x„,...,x  }.  There 

12       r 

are  3   clauses  in  C (r) .   This  is  because  the  definition  of 

clause  excludes  the  simultaneous  appearance  of  a  literal 

and  its  complement.   Thus,  for  each  of  the  r  atoms,  the 

atom  may  appear  in  a  clause  as  a  positive  literal,  negative 

literal   or  not  at  all,  generating  3   different  clauses. 

n 
Let  K(n,r)  =   X   C(r).   That  is,  an  element  of   K(n,r) 
i=l 
is  a  sequence  of  n  clauses   chosen  from  the  set  C(r).  There 

are  technical  advantages  to  defining  K(n,r)  as  a  sequence 

of  clauses  rather  than  as  a  set  of  clauses. The  clauses  are 

ordered,  allowing  for  the  appearance  of  identical  clauses. 

Furthermore,  there  is  a  natural  mechanism  that  extends  a 
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probability  density  function  (pdf)  from  C (r)  to  K(n,r), 

as  the  n-fold  product  space  of  C (r) .   The  process  in  which 

S  e  K(n,r)  is  chosen  randomly  according  to  this  induced  pdf 

can  be  envisioned  as  the  sequence  formed  by  making  n 

independent  choices  of  clauses  from  C(r)  according  to  the 

distribution  placed  on  C(r).   Thus,  if  P^,  AC  } ,    C^  g  C(r), 

i=l,...,n  is  the  probability  of  choosing  C.  from  C(r),  then 

the  probability  of  choosing  the  sequence  <C, ,...,C  >  from 

n 

K(n,r)  is  P{<C, , . . . ,C  >}  =  1  T  P^ ,  > {C. } .   (Nevertheless 
1      n      !_'C(r)i 

for  the  sake  of  ease  of  expression,  S  s  K(n,r)  will  sometimes 

be  referred  to  as  a  set      of  n  clauses.)   Let 

p.    =  Pp.  , {C  I  C  G  C{r)  and  x.  e  C},  i  =  l,...,r  ,  and 
'^i  ,r     C  (r)    '  1 

q.    =  Pp.  , {C  I  C  G  C(r)  and  x.  e  C},  i  =  l,...,r. 
^1 ,r     C (r)    '  1      ' 

In  the  absence  of  any  knowledge  regarding  the  relative 

frequency  of  problem  instances  and  of  the  underlying 

theoretical  distribution,  average  case  analyses  assume  that 

all  instances  are  equally  likely.   Hence  the  analysis  of 

the  DPP  will  proceed  by  first  assuming  that  the  uniform 

distribution  has  been  placed  on  C{r)  and,  consequently, 

on  K (n,r) . 

Lemma  2.1.   If  the  uniform  distribution  has  been  placed 

on  C  (r)  ,  then   p  •  „  =   q-  „  =  1/3,  i  =  1 ,  .  .  .  ,  r. 

X  ,  r      1 ,  r 

Proof.   For  any  atom  x.  ,  exactly  1/3  of  the  clauses  in  C(r) 

contain  x.  ,  and  exactly  1/3  contain  x. .  O 
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It  might  be  expected  that  obtaining  good  bounds  on 
the  complexity  of  the  DPP  would  be  contingent  upon  the 
frequent  applicability  of  the  unit  and  pure  literal  rules, 
since  it  is  the  splitting  rule  which  introduces  the  possi- 
bility of  exponentially  long  computations.   But  this  is 
not  the  case.   The  pure  literal  and  unit  rules  can  be 
regarded  as  special  cases  of  the  splitting  rule.   Let  the 
unit  rule  be  applicable  to  a  set  of  clauses  S  and  let  {L} 
be  a  unit  clause  in  S .   If  we   split  S  into  sets 
Sj^  =  {C  -  {L}  I  C  e  S  and  L  ^  C}   and 
S2  =  {C  -  {L}  I  C  e  s  and  L  ^  C},  then   0  e  S   , 
and  the  DPP  would  immediately  report  S„  unsatisf iable. 
Thus  application  of  the  unit  rule  on  literal  L  is  not 
essentially  different  from  application  of  the  splitting 
rule.   Next,  let  L  be  a  literal  that  appears  pure  in  S, 
i.e.,  C  n  {l}  -   0  for  every  C  e  S.   In  this  case,  applica- 
tion of  the  splitting  rule  on  L  creates  sets  S,  and  S_ 
with  S,  c  s_  .   Thus  the  pure  literal  rule  only  considers 
the  satisfiability  of  S, .   Therefore  the  point  of  view  taken 
here  is  that  for  the  DPP,  the  splitting  rule  is  fundamental. 
The  other  two  rules  amount  to  a  preference  for  splitting  on 
certain  literals  and  a  useful  optimization  —  detecting 
the  relation  S  '^  ^2'       Improved  performance  of  the  DPP  results 
from  minimizing  the  sizes  of  the  sets  S,  and  S„  .  In  effect, 
the  unit  and  pure  literal  rules  are  instances  of  the  appli- 
cation of  this  principle. 
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We  shall  analyze  a  procedure  DPP'  which  only  applies 
the  splitting  rule.  Furthermore,  DPP'  does  not  attempt  to 
make  a  strategic  choice  of  the  literal  on  which  the  set  is 
split.   It  instead  chooses  the  literal  at  random  giving 
equal  probability  to  each  literal  in  the  literal  set. 
The  algorithm  for  DPP'  follows: 

procedure    DPP'(S);  * 

1.  if  S   =   0   then   return    satisfiable  end   if; 

2.  i/  0  e  S  then   return    unsatisf iable  end   if', 

3.  Let  L  be  an  arbitrary  literal  from  i. 
S,  :=  {C  -  {L}  I  C  G  S  and  L  ^  C}  ; 
82:=  {C  -  {L}  I  C  6  S  and  L  ^  C}; 

if   DPP'(S,)  =  unsatisf iable  and 
DPP'(S2)  =  unsatisfiable 
then   return   unsatisfiable; 
else    return        satisfiable;  end   if; 
end   DPP  '  ; 

Lemma  2.2.  Let  S  be   a  set  of  clauses  chosen  uniformly 

from  C{r).  Let  S,  and  S^  be  defined  as  above.  Then  the 

clauses  of  S,   and  S^  are  each  uniformly  distributed 

over  C (r  -  1) . 

Proof.  For  each  clause  C  e  C(r  -  1),  there  are  precisely 
two  clauses,  namely  C  and  C  u  {L}  of  C(r)  whose  presence 
in  S  will  result  in  the  occurrence  of  C  in  S^.  So  the  probability 
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that  C  is  in  S^  is  equal  to  the  probability  that  either 

C  or  C  u  {l}  is  in  S.   Since   clauses  in  S  are  chosen 

uniformly  from  C(r),  for  any  C  ,C   G  C(r  -  1), 

P{C^  G  s^}  =  P{Cj^  G  S  or  C^  U  {L}  G  S}  =  I>{C^   G  s   or 

Cy   u  {L}  G  s}  =  ^{^2  e  s  }.   The  same  proof  applies  to 

S_  as  well.  D 

Lemma  2.3.   Let  S  G  K(n,r).   Then  for  some  constant  c,  c  >  0, 
cnr  bounds  the  time  needed  to  perform  one  step  of  DPP  or  DPP'. 
(A  step   of  DPP  tests  if  S  is  empty,  if  the  empty  set  is  in  S, 
if  the  unit  or  pure  literal  rule  can  be  applied,   and  then 
applies  the  appropriate  rule.) 

Proof .   Since  each  clause  contains  at  most  r  literals,  nr 
bounds  the  length  of  S.   In  one  scan  through  S  the  tests  of 
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the  DPP  and  DPP'  can  be  performed.   The  sets  S,  and  S_  can 
be  created  in  a  second  pass. 

If  the  steps  of  DPP  could  not  be  performed  in  linear 
time,  it  would  not  be  possible  to  infer  that  the  expected 
time  of  the  DPP  is  bounded  by  the  expected  time  of  DPP'. 
Subsumption    is  a  rule  of  inference  based  on  the  fact  that 
if  C,  and  C„  are  in  a  set  of  clauses,  S,  and  C,  c  C2  then 
S  is  unsatisf iable  iff  S  -  {C_}  is.   The  rule  consists  of 
deleting  all  such  "subsumed"  clauses   C- •   After  a  set  S  is 
split  into  sets  S,  and  S^   it  might  be  worthwhile  to  perform 
subsumption  and  possibly  delete  clauses  from  S,  or  S„. 
Unfortunately,  it  is  not  clear  that  subsumption  can  be  applied 
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in  linear  time.   When  subsumption  is  employed,  an  analysis 
like  the  one  below  will  show  that  the  procedure  has  poly- 
nomial expected  time  performance.   However,  the  additional 
overhead  of  subsumption  will  degrade  the  bound. 

Let  T(n,r)   be  the  expected  time  complexity  of  the  DPP' 
with  the  uniform  distribution  on  K(n,r). 


Theorem  2 . 4 


n-1 
cnr  +  J 
i=l 


n 
.i. 

2^ 

i3j 

1 

ril 

l3j 

n-i 


T(n,r)  =  ■ 


0 


T(i,r-1) ,  n,r>0 


otherwise 


Proof.   By  the  definition  of  the  expected  time  complexity 
of  an  algorithm  , 

T(n,r)  =     y      P{S}[the  time  DPP'  takes  on  S] . 
SGK(n,r) 

Given  S  e  K(n,r),  DPP'  first  performs  one  step  of  the  algo- 
rithm.  This  step  may  cause  the  algorithm  to  terminate  (in 
lines  1  or  2  of  the  algorithm)  or  it  may  result  in  the 
recursive  application  of  the  DPP'  to  sets  S^   and  S^.  By 
assuming  that  the  latter  occurs,  the   expected  time  complexity 
is  being  bounded.    If  the  algorithm  finds   that  S^    is 
satisfiable   it  does  not  have  to  check  the  satisfiability 
of  S  .   We  assume  the  algorithm  tests  S,  and  S   for  satis- 
fiability. 

The  time  needed  to  perform  one  step  of  DPP'  is  cnr 
(Lemma  2.3).   After  performing  that  step  the  algorithm  is 
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left  to  test  the  satisfiability  of  S,  and  S„.  Let  S,eK(n, ,r, ), 
SpG  K(n_,r^),  where   1  <  n,  <  n,  1  ^  n   <  n,  and  r  =  r  =  r-1. 
Since  clauses  of  S  are  uniformly  distributed  over  C(r),  by 
Lemma  2,2,  clauses  of  S   and  S„  are  uniformly  distributed 
over  C(r  -  1).    Thus,  T(n  ,r-l)  (respectively,  T(n2,r-1)) 
represents  the  expected  time  needed  to  check  the   satisfi- 
ability of  S   (respectively,  S^) .   Therefore, 

T(n,r)  =     I  cnr  +  T(n,,r-1)  +  T(n„,r-1)  . 

SeK(n,r) 

By  collecting  terms  involving  T(i,r-1),  l^i<n,  and  noting  that 

J^  P{S}cnr  =  cnr,  the  following  recursion  is    obtained: 

seK(n,r) 

n-1  r  >, 

T(n,r)  =  cnr  +  I      PiS  splits  so  that  | S, |  =  ir   T(i,r-1) 

n-1  f  ■\ 

+      I      PJS  splits  so  that  |  S^  |  =  ij'T(i,r-l)  . 


However , 

P-js  splits  so  that  |  S,  |  =  i 


n 
i 

2 
3 

1 

[1] 
3^ 

n-1 


since   for  each  clause,  with  probability  2/3   (Lemma  2.1), 
that  clause   (with  L  deleted)   is  put  into  S, .   Since 
the  same   probability  applies  to  S„  we  get 


T(n,r)  =  ■ 


n-1 
cnr  +2  1 
i=l 


n 
i 

\2' 
l3j 

1 

I3j 

n-1 


T(i,r-1)  ,   n,r  >  0 

otherwise. 
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Lemma  2.5.   T{n,r)  <  crT'(n),  where 

,^  i 


n-1 

T'  (n)  =  n  +  2  J 
i=l 


n-i 


T'  (i) 


Proof.   The  proof  is  by  induction.   The  base  case,  n 
is  trivial.  Assume  the  lemma  true  for  i  <  n. 


=  0, 


n-1  r 
T(n,r)  =  crn  +2  J 

i=l 


n-1 


T(i,r-i: 


Applying  the  induction  hypothesis, 

r„\  i 


n-1 

T  (n,r)     <_  crn   +2       I 

i=l 


n 

liJ 


2 

I3j 


1 

13J 


n-1 


c(r   -    1)    T' (i) 


Since   all  terms  are  positive. 


n-1 
<_   crn  +  2  I 
i=l 


n 
i 

'2 

1 

l" 

3^ 

n-1 


crT' (i) 


=  crT'  (i; 
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Our  goal  is  to  bound  T(n,r).   The  previous  lemma 
simplified  this  task  by  allowing  us  to  consider  a  function 
of  one  variable,  T' (n) . 


Theorem  2.6.   T(n,r)  <^  crn  . 

2 
Proof.   It  suffices  to  show  that  T'(n)  <_  n  .   This  will  be 

done  by   induction.   Direct  computation  of  T'(n)  verifies 

the  assertion  for  n  =  1,...,13.   This  will  serve  as  the  base 

case  of  the  induction. 


n-1  r 
T'  (n)  =  n+2  I 
i=l 


3 


n-1 


T-  (i) 
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By  induction, 

T'(n)  <  n+2   J    "    I    \ 
j^  =  \     >.^J  wJ   W- 

2 
which  must  be  shown  to  be  less  than  n  . 

Extending  the  sum  with  positive  terms. 


.2 

1 


n 


n 


The  sum,  J 
i=0 


T' 

(n) 

<    n+2 

. 

i  =  0 

n 

i_ 

.3j 

i 

[1^ 
13J 

n-i 

->  n-1 


i   ,  is  the  second  moment  of 

the  binomial  distribution  with  parameter  p  =  2/3.  This  can 

be  shown  equal  to  [Davis,  1962] 

2 
p  n(n  -  1)  +  np 

(Note  that  this  second  moment  differs  from  the  variance, 
np(l  -  p) ,  which  is  the  second  moment  centered  around 
the  mean.)   Substituting,  we  get 


T' (n)  <  n+2 


g  n  ( n  -  1 )  +  2  i^ 


8   2  ^  13 
=  9  n   +  -^n 


Observing  that  because  the  coefficient  of  n   in  this  poly- 

2 


nomial  is  less  than   1,  we  have   for   n  ^  13,  (8/9)n  +(13/9)n 
<  (8/9)n^  +  (l/9)n^  -  n^  .  □ 

To  strengthen  our  claim  that  the  satisfiability  problem 
is  tractable  when  the  DPP  is  employed,  we  now  show  how  to 
obtain  polynomial  bounds  when  distributions  other  than 
the  uniform  distribution  on  the  problem  instances  are  assumed, 
As  above   we  shall  still  consider  K(n,r)  as  the  n-fold 
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product   space  of  C(r).  Furthermore,  an  analogue  of  Lemma  2.2 

must  be  valid  for  the  distributions  placed  on  C (r) .  That  is, 

the  removal   of  an  arbitrarily  chosen  literal  from  clauses 

of  C(r)  distributed  according  to  the  pdf  on  C(r)  must  result 

in  clauses  of  C (r  -  1)  distributed  according  to  its  pdf. 

This  condition  is  a  natural  one  and  would  only  fail  if 

radically  different  distributions  are  put  on  C(r)  and  C(r-l). 

Finally,  we  must  assume  that   inf  {p.  ,q.  }  =  q  is  greater 

i,r 
than  0.   This  condition  is  equivalent  to  stating   that  for 

any  literal  L,  the  probability  of  its  appearance  in  a  clause 

in  C(r)  is  bounded  away  from  zero  as  r  ^  «>• 

Theorem  2.7.   Let  p  =  1-q.   Under  the  conditions  on  the  pdf 
for  C(r)  above,  the  expected  time  of  DPP'  is  bounded  by 


n-1 
T  (n,r)  =  cnr  +2  I 
P  i=l 


p^  q"^  ^  Tp(i,  r  -  1) 


Proof.    The  proof  is  identical  to  that  of  Theorem  2.4 
In  time  cnr  one  step  of  the  DPP'  is  performed  leaving 
two  problems  each  of  size  i  with  probability 


1   n-i 

p  q 


(Actually  when  removing  x.  from  clauses  in  C(r)   the 
probability  of  obtaining  a  split  set,  S,  ,  of  size  i  is 


1 
n 
i 


1  -  p  .  I   p.     .     Hence,  we  are  overestimating  the 


size  of  the  split  set  by  using  p  and  q) ,  g 

The  uniform  distribution  corresponds  to  p  =  2/3. 

The  expected  size  of  a  split  set  is  np.  As  p  ->-  1 ,  the  result 

of  a  split  is  less  effective  in  terms  of  reducing  the  size 
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of  the  split  sets.   This  increases  the  expected  time  behavior 
of  the  algorithm.   Let 


n-1  f    \ 
T' (n)  =  n+2   T    ^ 
P  i  =  l 


p  q    T  1 

^      p 


Clearly , 


T  (n,r)   <   cr  Tp(n) 

n-1 
Lemma  2.8.   Let   f   ^ (n)  =  n+2   J 

i=l 


n 


"p,k 


i   n-i   k   , 
p   q     n   where 


k  is  integral  and  0<p<l.    Ifk>  -1/log-jP 
then   f   v(n)  <  n  ,  for  sufficiently  large  n. 

p,K      — 


Proof 


The  sum 


..<"'  i-^  J„  [l 


i   n-i   k 
P  q    n 


n 

I 
i=0 


i   n-i   k  .   .  ,   ,  th 
p   q     n   IS  the  k    moment 


of  the  binomial  distribution.   By   use  of  finite  difference 
methods  [Riordan,  1958]   it  can  be  shown  that 


n  r 

I 
i=0 


1   n-1  .k     V 
P  q    1   =   Z 
s=l 


k    (s)   s 
n     p   , 


ft} 


denotes  Stirling  numbers  of  the  second  kind  and 


where 

(s) 
n     =  n(n  -  1)  ...  (n  -  s  +  1).   This  sum  is  a  polynomial 

]^ 
in  n  of  degree  k.   The  coefficient  of  the  n    term 

1  we  have 


.kl   k    ,   .     /k\ 
is  ■<,_)■  p   and  since  ■{,>   - 


k  k 


f   ,  (n)  <  n  +  2 (p  n   +  lower  order  terms  in  n) 

p  ,  K 

k  k 
<  2p  n   +   lower  order  terms  in  n  . 


Hence,  for  sufficiently  large  n, 

f 


■p,k 


(n)  <  n^  «  2p'^  <  1. 


p^  <  1/2 


k  logjP  <  -1 


<*  k  >  - 


log^p 


,  since  log2P  <  0 


D 
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Thus  we  expect  that  the  following  result  is  obtainable; 


T  (n,r)  <^  crn    where    k  = 
P  log2P 

Completion  of  an  inductive  proof  of  the  statement  for  a 
particular  p  requires  establishment  of  the  base  case; 

Lemma  2.8  then  supplies  the  inductive  case.   This  involves 

I        k 
verifying  that  T  (n)  <_   n   for  all  n  less  than  some  N^  , 

where   N   can  be  computed  as  in  Theorem  2.6 


However,  for  p  such  that 


log2P 


is  small 


log2P 

such  a  verification  may  not  be  possible.   For  these  p, 

,  k+1 

we  must  settle  for  T  (n)  bounded  by  n 

P 

As  p   ->-  1 ,  log„p  ->■   0   and  so  k  gets  large.  The  growth 

of  k  as  a  function  of  p  is  tabulated  in  Figure  2.1.  A  table 

of  the  function  T' (n)  for  n  =  1  to  n  =  50  and  various  values 

P 

of  p   is  Figure  2,2.   Figures  2.3,  2.4,  2.5,  2.6  and  2.7  graph 
t' (n)  for  p  =  0.5,  0.6,  0.7,  0.8  and  0.9. 

It  is  helpful  to  consider  a  simpler  equation  that, 
while  not  adhering  to  the  definition  of  the  expected  time 
complexity  of  an  algorithm,  does  capture  in  a  simplified 
context   the  essence  of  the  argument  made  above.  Let 


C  (n,r)  = 
p 


cnr  +  2C  (pn,  r  -  1) ,    n,r  >  0 
0  otherwise 


for   0  <  p  <  1.   This  equation  simplifies  T  (n,r)  by  not 
summing  over  all  possible  sizes  of  the  split  sets,  but 
considers  only  the  expected  size  pn  of  a  split  set. 
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Figure    2.1 
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Figure  2.7 
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Interestingly,  it  is  easy  to  show  that  both  equations  give 
the  same  complexity  bounds . 


Theorem  2.9.   C  (n,r)  is  0(rn  )  where  k  =  -  -, 
p  ±092? 


Proof.   C  (n,r)  =  cnr  +  2  C  (pn,  r-i; 


(n,r  >  0) 


=  cnr  +  2(cpn(r-l)  +  2(cp  n(r-2)  +  ...)) 

2   2       3   3 
<  cnr  +  2cpnr  +  2  cp  nr  +  2  cp  nr  +  ... 


m 


=  cnr  I       (2p) 
i=0 


where  m  satisfies  the  condition 

m     T 
p  n  =  1  I.e. 


m  =  -  log   n 


,    (2p)l-'"^  -  1 

I       (2p)^  =  

i=0  2p  -  1 


C  (n,r)  <  crn 
P 


1-log  n 
(2p)      P   -  1 

2p  -  1 

-log  n 


<  ^^P^  ,  rn  (2p) 
—  2p  -  1 


-log  n 


=  c'r2 


-log  2 

1       P 
=  c ' rn     ^ 


.  ^   2pc 
2p  -  1 


-l/log„p 


=   c  rn 


which  is   0 (rn  )  . 
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The  variance  of  the  time  complexity  of  an  algorithm, 
AL,  measures  the   deviation  of  the  running  time  of  AL  on  an 
arbitrary  problem  instance  from  the  expected  running  time 
of  the  algorithm.   If  the  variance  is  small,  then  instances 
on  which  the  algorithm's  running  time  is  significantly 
slower  (or  faster)  than  the  expected  running  time  would  be 
rare.   The  variance  of  the  DPP  has  not  been  computed.  But 
there  is  evidence  that  the  variance  is  small.   In  effect, 
C  (n,r)  measures  the  complexity  of  DPP'  under  the  assump- 
tion that  the  variance  is  zero.   Since      C  (n,r)  grows  at 

the  same  rate  as  T  (n,r),  it  is  reasonable  to  believe  that 

P 

the  variance  of  T  (n,r)  is  not  large.  Second,   it  is 
clear  from  examination  of  DPP'  that  the  variance  of  the 
complexity  is  due  to  deviations  in  the  size  of  the  split 
sets  from  the  expected  size.   Thus,  the  variance  of  the 
complexity  of  DPP'  is  closely  related  to  the  variance  of 
the  binomial  distribution,  since  the  size  of  a  split  set 
is  distributed  according  to  this  distribution.  If  X  is  a 
binomially  distributed  random  variable,  with  parameters 
n  and  p,  then  by  Chebyshev's  inequality,  for  £  >  0, 

P{|X  -  np|  >  e}  <  ^P^^  -  P^  . 
So  for  any  fixed  k , 

p{|x  -  npl  >  ^1  <  ^P^^   -   P^^^    =    ^^   -  P)^^^ 

which  approaches  0  as  n  -»■  «>.   Thus,  the  probability  that 
a  split  set  deviates  from  its  expected  size  by  more  than 
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a  fixed  percentage  approaches  0  as  n  ^-  «>.   This  helps 

explain  the  relevance  of  C  (n,r)  to  T  (n,r). 

P  P 

It  was  hoped  that  an  explicit  solution  to  the  recur- 


rence for  T  (n,r)  would  be  derived.   Towards  this  end, 
the  generating  function  of  T  (n)  was  obtained.  Starting 

3  J  P  J 

with  the  defining  relation  for  T'(n), 


n-1     f 

n    +    2       I 

Tp(n)     = 

i  =  l    ^ 
0 

p   q     T(i)  ,   n  >  0 


n  =  0 


we  extend  the  sum  to  form  the  complete  convolution, 


n 


(1  +  2p")  t' (n)  -  n  +  2   J 


i=0 


p"  q""'  T'(i) 


T' (n)       ,         n    n-i       .  T  (i; 

(1  +  2p")  -^^y—  =   -r-4^   +  2  I      ^—r—   .  p^  -P^ 
'^     n!      (n-1)!      .  ^^  (n-i)  !    ^     i! 


t'  (n) 
Let   f(n)  =  — 2-| an 


nl 


d  let  F(z)  =  I      f(n)  z"  be  the 
i=0 


generating  function  of  f(n).   In  the  notation  that  follows 
Gf{n)  =  F(z).   Observe  [Jordan,  1947] 


^    1  z 

^  (n  -  D!  =  ^  ^ 


n    n-1  n     n-i 


n 


=  2G  3-  G  p"f (n) 


n! 


=  2  e"^^    F(pz) 


G(l  +  2p'^)  f(n)  =  G  f(n)  +  2  G  p"  f(n)  =  F(z)  +  2F(pz) 
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Since  f(0)  =  0,  these  equations  are  valid  for  n  >^  0 .  Hence, 

F(z)  +  2F{pz)  =  ze^  +  2e'^^  F(pz) 

is  a  functional  identity  that  must  be  satisfied  by  F(z). 
Solving  for   F(z)   we  get 

F(z)  =  ze^  +  2(e^^-l)  F(pz)  . 

Expanding  the  previous  equation  by  substituting  pz  for   z 
yields 

F(z)  =  ze^  +  2(e'^^-l)  (pzeP^  +  2  (e^^^-l)  F(p^z))  . 


Performing  this  substitution  n  times. 


F(z)  = 


n-1     i 


e^P  ^  -  1 


i=0 


2^  F(p^z)  +  ""j       (2p)^  zeP^^  ^(e^P  "-1 
j=0  k=0 


It  can  be  shown  that  the  first  term  of  this  expression  does 
not  contribute  to  the  coefficient  of  z   ,  for  k  =  l,...,n. 
Even  with  this  simplification,  it  is  improbable  that  an 
explicit  expression  for  f(n)  can  be  obtained. 

We  next  present  a  result  which  shows  that  in  a  very 
strong  sense  the  instances  in  which  the  DPP  takes  exponential 
time  are  rare.   The  next  lemma  restates  the  result  of 
Theorem  2.6  in  a  weakened  form,  where  the  size  of  a  problem 
instance  is  used  to  parameterize  the  sample  space. 

Lemma  2.10.    Let  S    be  the  set  of  all  sets  of  clauses  of 
n 

size  (i.e.  length)  n.   Let  R(n)  be  the  expected  time  complex- 
ity of  the  DPP  with  the  uniform  distribution  as  the  underlying 

3 
distribution.  Then,  R(n)  £  en  . 
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Proof.   Any  formula  of  length  n  can  have  at  most  n  distinct 

atoms  and  n  clauses  and  so  is  in  K(n,n).  Therefore, 

3 
R(n)  £  T(n,n)  <^  en  .  ^ 

Let  X(-)  be  a  predicate  on  the  set  of    problem 

instances,   S  =  u  s   and  let  q  be  the  probability  that 

n 

X(I)  does  not   hold  when  I  is  drawn  from  S  .  Following 

n 

Karp  [Karp,  1976]  a  predicate  X  holds    almost   everywhere 

oo 

if   J^   q   converges.   This  condition  implies  that  if  we 

n=l 
draw  an  infinite  sequence  of  problem  instances,  one  of 

each  size,  then  with  probability  1,  the  predicate  X  will 

fail  on  only  finitely  many. 

Lemma  2.11.    Let   f (n)   be  the  expected  time  complexity 
of  algorithm  AL  on  S   ,  problem  instances  of  size  n. 

Assume  that  f  (n)  <^  en  .   For  a  >  0,  let  X  (I)  be  the  predi- 

ctn 
cate  "AL  computing  on  I  e  s   takes  time  bounded  by  e   .  " 

Then   X  (I)  holds  almost  everywhere. 

a 


Proof.   f(n)  -      1  P{l} [running  time  of  AL  on  I] 

les  , 
n 

X(I)  true 


+  \  P{l} [running  time  of  AL  on  I ]  £  en  , 

les   , 
n 

X(I)  false 
Dropping  the  first  term  and  observing  that 

\  P{  I }  [running  time  of  AL  on  I]   >_  q   e 

IGS   ,  -   n 

n 

X{I)  false 

the  following  inequality  is     obtained: 
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an 


1  ^ 

0<qe         <cn  orO<q<    . 

—  ^n     —  —  ^n  —  an 

e 

So  if  l      — —  converges,  then    1      q   will  converge. 
n=0  e  n=0 

But  by  applying  the  ratio  test, 
„(n+l)^ 


lim 


a(n+l) 
e 


k 
en 

an 
e 


,  .   (n+l)'^  1   ,  .  (n+1)^  1   ,  , 

lim  -^ ^ —  =  —  lim  -^ —  =  <  1 

a  k      a       n  a 

n-^°°   en  e   n->-o°  e 


Corollary  2.12.   For  any  a  >  0,  the  DPP  takes  time  less 

an 
than  e     almost  everywhere  on  a  set  of  clauses  of 

length  n. 

Proof.   Apply  the  two  previous  lemmas. 
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Performing  an  analysis  of  an  algorithm  often  yields 
deeper  insights  into  the  nature    of  the  algorithm. 
The  analysis  of  the  DPP   reveals  that  the  worst  case 
behavior  occurs  for  sets  of  clauses  on  which  the  splitting 
rule  does  not  effectively  reduce  the  size  of  the  set. 
This  in  itself  suggests  a  strategy  for  choosing  the  literal 
on  which  splitting  is  to  occur,  namely,  choose  the  literal  L 
tnat  occurs  most  frequently  in  the  set.   This  rule  can  be 
implemented  ecnomically.   For  each  literal  pair,  L  and  L, 
counts  of  their  occurrences  in  the  set  are  easily  maintained. 
Other  factors  can  be  incorporated  into  this   strategy. 
For  example,  splits  that  allow  the  pure  literal  or  unit 
rule  to  be  applied  subsequently  may  be  favored.   The  analysis 
also  warns  that  the  literal  selection  procedure  should  not 
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be  too  complicated;  the  selection  should  be  made  in  linear 
time.   The  usefulness  of  subsumption  is  also  called  into 
question,  since  one  step  of  the  DPP  may  not  take  linear 
time  if  subsumption  is  employed. 

Reducibility  (see  Ch,  I)  is  an  important  notion 
employed  in  the  study  of  the  worst  case  behavior  of 
an  algorithm.   If  problem  A  can  be  polynomially  reduced 
to  problem  B  and  a  polynomial  (worst  case)  exists  for  B, 
then  one  exists  for   A  as  well.   Unfortunately,  such 
arguments  cannot  be  made  in  the  average  case.  Instances 
of  problem  A  are  typically  mapped  by  the  reduction  into  a 
subclass  of  the  instances  of  problem  B,  violating  the 
distributional  assumptions  underlying  the  analysis.  Reduc- 
tions frequently  map  problem  instances  into  sets  of  clauses 
in  •Buch  a  way  that  as  the  size  of  the  problem  instances 
increase,  the  probability  that  a  literal  appears  in  a  clause 
in  the  corresponding  set  goes  to  zero.   This  violates  one  of 
the  conditions  required  of  the  distribution  in  order  for 
the  average  case  analysis  to  be  applicable. 

This    of  course  does  not  in  any  way  imply  that  other 
WP-complete  problems  may  not  have  algorithms  which  have 
polynomial  average  case  behavior.   We  believe  that  at  least 
some  of  them  do.   However,  it  is  the  "divide  and  conquer" 
nature  of  the  DPP  that  enables  us  to  perform  this  analysis. 
Algorithms  of  this  type  should  be  sought  for  other  NP- 
complete  problems. 
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III.    PROOF  SYSTEMS  AND  THE  DAVIS-PUTNAM  PROCEDURE 

In  this  chapter,  the  relationship   of  the  Davis-Putnam 
procedure,  viewed  as  a  proof  procedure,  to  other  proof 
systems  for  the  prepositional  calculus   Ls  examined.   Reckow, 
in  his  thesis  [Reckow,  1976],  assesses  the  relative  power 
of  many  classical  proof  systems.   If  proof  system  A 
p-simulates  B   then  A  is  at  least  as  powerful  as  B.  His 
investigation  shows  that  Frege  systems  are  the  most  powerful 
since  they  can  simulate  all  other  systems.   Gentzen,  natural 
deduction,  sequent,  full  resolution  and  extended  regular 
resolution  are  of  roughly  comparable  power.   Exponential 
lower  bounds   on  proof  length  have  not  been  established 
for  these  systems.   Recall,  that  if  MP   i-    WP   then  all  proof 
systems  are  not  polynomially  bounded.   Analytic   tableaux 
and  regular  resolution  have  known  exponential  worst  cases, 
but  are  still  superior  to  full  truth  tables.   The  Davis-Putnam 
procedure  has  also  been  classified  by  Reckow  in  the  partial 
ordering  of  proof  systems  determined  by  the  p-simulates 
relation.   However,  the  Davis-Putnam  procedure  considered 
in  his  study  does  not  correspond  to  what  has  been  called 
the  DPP  in  this  thesis.   The  difference  can  be  traced  to 
historical  reasons.   In  1960,  the  Davis-Putnam  procedure 
was  introduced  in  a  paper  called  "A  Computing  Procedure 
for  Quantification  Theory"  [Davis  and  Putnam,  1960] . 
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The  unit  clause  rule  and  the  pure  literal  rule  were  intro- 
duced.  However,  in  place  of  the  splitting  rule  there  is  a 
rule  called  "rule  for  eliminating  atomic  formulas": 
Write  a  formula,  (^ ,    given  in  CNF  as  (A  V  L)  &  (B  v  L)  &  R, 
for  some  literal  L.   That  is,  choose  L,  and  collect  all 
clauses  containing  L.   Then  factor  L  from  those  clauses 
leaving  a  conjunction  A.   B  is  the  conjunction  of  clauses 
obtained  by  factoring  L  from  all  clauses  containing  L, 
R  is  the  conjunction  of  clauses  not  containing  L  or  its 
complement.   Then  4)  is  satisfiable  iff  (A  V  B)  &  R  is 
satisfiable.   The  formula  (A  V  B)  &  R  is  then  restored  to  CNF 
and  a  new  rule  is  applied.   The  rule  for  eliminating  atomic 
formulas  is  sound  and  complete;  the  procedure  terminates 
when  either  the  empty  clause  is  derived  or  the  set  becomes 
empty.   The  difficulty  with  this  rule  is  that  restoration 
of  (A  V  B)  &  C  into  CNF  requires  distributing  A  through  B 
which  frequently  results  in  a  combinatorial  explosion  of 
new  clauses   added  to  the  set.   Many  of  these  clauses  are 
tautologies  or  subject  to  subsumption.   The  connection  of 
this  rule  with  resolution  is  evident;  the  set  of  clauses 
obtained  by  putting  (A  V  B)  &  R  into  CNF  is  precisely  the 
set  obtained  by  forming  all  resolvents  of  clauses  of  <p    in 
which  L  is  the  annihilated  literal,  and  then  dropping  all 
clauses   containing  L  or  its  complement.   Thus,  this  version 
of  the  Davis-Putnam  procedure  can  be  described  as  a  resolu- 
tion procedure  which  we  call  DPP": 
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procedure    DPP"(S); 
repeat    forever; 

1.  if   S   =   0   then    return         satisfiable  end    if; 

2.  if   0  G    S    then   return    unsatisf iable  end    if; 

3.  Choose  a  literal  L  appearing  in  S. 
T:={C  |CGS&LGC}; 

R:=  {C^  -  {L}  U  C^  -  {L}  |  C^  6  T  &  L  e  C^^  &  C^eT    &  LGC2 }  ; 
S  :=  S  -  T  U  R  ; 

4.  Remove  all  subsumed  and  tautological  clauses; 
end   repeat  ; 

end   DPP"  ; 

Step  4  is  optional.   This  algorithm  can  be  adopted  to  include 
the  unit  and  pure  literal  rules.   The  algorithm  as  presented 
has  been  the  basis  of  all  the  complexity  work  done  on  the 
Davis-Putnam  procedure  [Cook,  1971;  Cook  and  Reckow,  1974; 
Galil,  1977;  Simon,  1971].   Fortunately,  the  results  reported 
by  the  above  researchers  do  not   depend  upon  the  exclusion 
of  the  two  rules . 

In  1962,  Davis,  Logemann  and  Loveland  [Davis,   Logemann 
and  Loveland,  1962]  found  the   rule   for  eliminating  atomic 
formulas  increased  the  number  and  lengths  of  the  clauses  too 
quickly  in  their  theorem  prover.   Thus,  they  replaced  it 
with  the  splitting  rule.   In  addition  to  keeping  the  number 
of  new  clauses  generated  manageable,  it  also  increased  the 
frequency  of  application  of  the  unit  clause  rule. 
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One  of  the  few  similarities  between  DPP  and  DPP"  is 
that  they  both  eliminate  every  occurrence  of  an  atom  in 
each  application  of  a  rule.   As  a  result  they  are  both 
regular  resolution  procedures.   It  has  just  been  shown 
how  to  regard  the  196  0  procedure  as  a  resolution  procedure; 
we  now  do  the  same  for  DPP. 

We  will  first  show  how  to  modify  the  DPP  procedure  to 
generate  resolution  refutations  with  only  a  constant  factor 
increase  in  the  running  time  of  the  procedure.  The  result- 
ing resolution  refutation  will  be  regular.   A  closer 
examination  of  the  proof  will  reveal  what  is  the  proper 
resolution  analogue  to  DPP.     The  recursive  structure 
of  the  procedure  will  be  used  to  recursively  generate  the 
resolution  refutation.   Suppose  the  DPP,  given  an 
unsatisfiable   set    S,   splits    S    on 

literal  L  into   sets  S   and  S  .   Since  S   and  S   are  each 
unsatisfiable,  the  DPP  can  inductively   generate  resolution 
refutations  for  these  sets.   Restoring  the  literal  L  (L)  into 
the  appropriate  clauses  in  the  derivation,  refutations 
from  S,  (S  )  yield  resolution  derivations  of  L  and  L  from  S. 
Resolving  L  and  L  derives   the  empty  clause  and  so 
completes  the  construction  of  the  refutation  from  S. 
The  complete  algorithm  is  described  as  the  procedure 
RES  (S,  deriv)  . 
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procedure    RES (S ,deriv) ; 
^/  S  =  0  then   deriv  :=  (  ) ; 

return         satisf iable;  end    if; 
•£/  0  G  S  then    deriv  :=  (0)  ; 

return   unsatisf iable;  end   if; 
if   the   pure  literal  rule  applies  to  S  then 
if   RES (S,, deriv)  =  satisf iable 
then   return        satisfiable; 

else    return    unsatisf iable; 
end   if; 
end    if; 
if    the  unit  rule  applies  to  S  then 

if   RES (S, , deriv, )  =  satisfiable 
then    return    satisfiable; 
else    deriv  :-  (restore  L  to  clauses  of  deriv, 
(yielding  a  derivation  of  L)  append  a 
resolution  with  L) ; 
return    unsatisf iable; 
end    if; 
else     (the  splitting  rule  applies)  if 
RES (S,, deriv, )  =  satisfiable  or 
RES (S„ ,deriVp)  =  satisfiable  then 
deriv  :=  (  ) ;  return    satisfiable; 
else    deriv  =  (restore  L  to  clauses  of 
of  deriv,  ,  restore  L   to  clauses  of  deriv2  , 
append  them  together,  resolve  L  with  L) ; 

return    unsatisf iable;    end   if; 
end    if; 
end   RES; 

57 


Theorem  3.1.   There  is  an  algorithm  that  generates  resolu- 
tion refutations  of  an  iinsatisf iable  set  of  clauses,  S, 
whose  running  time  is  within  a  constant  factor  of  the 
running  time  of  the  DPP. 

Proof.   The  above  procedure  may  not  achieve  the  bound 

specified  by  the  theorem  because  of  the  necessity  of  scanning 

the  derivation  repeatedly  to  restore  deleted  literals.  This 

problem  is  easily  overcome   by  initially  inserting  the  complete 

clause,  without  any  of  its  literals   deleted,  into  the 

derivation.   Hence  a  derivation  is  built  from  constituent 

derivations  in  step  5  by  appending  the  two  together  and 

adding  an  additional  resolution  between  the  derived  clauses. 

The  refutation  constructed  in  this  way  will  be  identical 

to  the  one  generated  by  RES. 

D 

In  this  chapter  our  concern  is  with  proof  length;  that 
is  our  measure  of  strength  of  proof  systems.  We  have   not, 
as  yet,  described  what  a  DPP  proof  is.   The  definition  of 
proof  requires  that  there  be  an  algorithm  which  verifies 
the  validity  of  a  purported  proof  in  polynomial  time. 
We  shall  consider  a  DPP  proof   to  be  a  tree  with  nodes 
labeled  with  the  sets  derived  by  application  of  the  rules 
in  the  execution  of  the  procedure.   The  given  set  is  placed 
at  the  root.   It  has  either  one  or  two  descendants,  depend- 
ing upon  which  rules   have  been  applied.   The  descendant 
sets  S,  and  S^  are  modified  so  that  literals  deleted  from 
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clauses  are   only  marked  as  such.   This  facilitates  the 
generation  of  the  resolution  refutation.  Alternatively, 
a  DPP  proof  could  be  given  as  a  tree  in  which  each  node  con- 
tains the  literal  eliminated   at  the  subsequent   step  of 
the  procedure   together  with  the  original  set  of  clauses. 
This  puts  the  burden  of  reconstructing  the  sets  on  the 
proof  checker.   This  can  be  done  in  polynomial  time  in  the 
size  of  the  tree  and  the  size  of  the  original  set.   It  is 
easily  checked  that  both  of  these  descriptions  are  reason-       { 
able  notions  of  proof,  consistent  with  the  definition  of 
proof  system.   Given  a  DPP  proof  as  originally  described 

(i.e.  with  sets  at  each  node),  an  endorder   traverse        ,  t, 

\  li- 
on the  tree  can  turn  it  into  a  regular  tree  resolution    (-'     ,  ? 


,..v  ex  '  - ■ 


proof.   This  is  done  by  the  following  procedure: 


...(^  *^ 


procedure    TREE -PROOF (DPP-TREE) ; 

1.  Let  S  be  the  set  of  clauses  at  the  root  of  DPP-TREE; 

2.  if   DPP-TREE  has  a  single  node  then 

restore  the  deleted  literals  to  the  empty  clause  in  S; 
label  the  node  with  this  clause; 
return;    end   if) 

3.  if   the  root  node  of  DPP-TREE  has  two  descendants, 
N,  and  N^  ,  then 

TREE-PROOF (N  ) ;  /*  endorder  traverse  */ 

TREE-PROOF (N-) ; 

label  the  root  node  with  the  resolvent  of  the 

clauses  labeling  its  descendants; 

return;    end    if; 
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4.    if   the  root  node  has  one  descendant,  N,  then 

if   the  descendant  was  obtained  by  the  unit  rule  then 
TREE-PROOF (N  ) ; 

Add  the  unit  clause  of  S  with  its  deleted 
literals  restored  as  a  new  descendant  of  the 
root  ; 

label  the  root  with  the  resolvent  of  the  2 
clauses  labeling  the  descendants; 
return;       end    if', 
else      /*  the  descendant  was  obtained  by  the  pure 

literal  rule  */ 
TREE-PROOF (N  ) ; 

delete  the  root  node.  Its  ancestor  will  link  up 
with  its  single  descendant. 
return;       end    if; 
end    TREE-PROOF; 

This  procedure  is  performing  an  endorder  (left  descendant, 
right  descendant , root)  traversal  of  a  DPP  tree  replacing 
the  sets  labeling  the  nodes  with  the  clauses  of  a  regular 
tree  refutation.   At  leaf  nodes,  a  clause  from  the  original 
set  replaces  the  set  of  clauses  at  that  node.  The  clause 
chosen  is  the  one  which,  in  the  execution  of  the  DPP,  has  had 
all  its  literals  dropped.   Once  the  leaf  nodes  are 
specified,  the  clauses  labeling  nonleaf   nodes  are  computed 
as  the  resolvents  of  its  descendants.   Application  of  the 


60 


pure  literal  rule  does  not  add  a  resolution  to  the  proof 
since  its  effect  is  to  locate  an  unsatisf iable  subset  of 
the  original  set.   The  proof  of  the  correctness  of  this 
procedure  is  based  on  the  proof  of  the  previous  theorem. 
Figure  3.1   is  an  example  of  a  DPP  refutation  and  the 
corresponding  regular  tree  refutation. 

Theorem  3.2.    Regular  tree  resolution  p-siraulates  the  DPP. 

Proof.   The  procedure  TREE-PROOF  does  the  simulation.  By 
construction,  the  resulting  resolution   refutation  is  a 
tree.   Because  the  DPP  completely  removes  the  occurrence 
of  an  atom  in  the  derived  sets,  the  resolution  refutation 
is  regular.  D 

It  has  been  remarked  in  Chapter  I  that  a  difficulty 
with  tree  resolution  is  the  repetition  of  the  derivation 
of  any  clauses  needed  more  than  once  in  a  derivation.  DPP 
has  this  problem  as  well  since  a  split  may  be  performed 
yielding  two  sets  and  the  same  clause  may  have  to  be 
derived  in  both  of  these  sets. 

Theorem  3.3.   DPP  p-simulates  regular  tree  resolution. 

Proof.   Consider  the  alternate  description  of  a  DPP  proof 
as  a  tree  whose  nodes  are  labeled  with  the  literal  annihilated 
at  the  subsequent  step  of  the  algorithm.  Since  we  can 
construct  a  standard  DPP  proof  from  this  version  in  polynomial 
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DPP  REFUTATION 

{rpn,  r£,  pi,    £m,  £q,  mqn ,  n} 

(unit  n) 

_         I         _ 
{rpn,  rH,    p£,  £m,  £q,  mqn} 

(split  2) 

{rpn,  rl,    p£ ,  mqn}                {rpn,  £m,  £q,  mqn} 

(pure  m)  (unit  m) 

{rpn,  r£,  p£ }                    {rpn,  £q,  mqn} 


{rpn,  p£} 


{p£} 


(unit  r) 


(unit  p) 


(unit  q) 


{ rpn ,  mgn } 


TREE  RESOLUTION  REFUTATION 


Figure  3.1.  Underlined  literals  have  been  marked  deleted 
in  DPP  refutation. 
(Example  taken  from  [Shostack,  1976].  ) 
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time,  it  is  only  necessary  to  show  how  a  regular  tree  resolu- 
tion proof  can  be  p-simulated  by  this  proof  scheme.  This  is 
accomplished  by  replacing  the  resolvent  clause  at  a   nonleaf 
node  by  the  literal  annihilated  in  forming  that  resolvent 
and  deleting  all  the  leaf  nodes.   It  is  easily  proved 
by  induction  that  the  resulting  tree  does  correspond  to  a 
correct  DPP  refutation.  r-. 

Refutation  graphs  were  defined  by  Shostack  [Shostack, 
1976]  as  a  convenient  formalism  from  which  to  study  theorem 
proving  methods.   It  is  easy  to  show  how  a  refutation  graph 
can  be  constructed  from  a  Davis-Putnam  refutation  in  linear 
time.   Shostack  does  this  implicitly  in  his  construction  of 
a  refutation  graph  for  unsatisf iable  sets  [Shostack,  1976], 
Thus,  refutation  graphs   p-simulate  the  DPP.  Given  a  refuta- 
tion graph,  other  types  of  resolution  refutations  can  be 
generated  in  polynomial  time.   Most  notable  among  these  is 
t-linear  resolution  [Kowalski  and  Kuehner,  1971].  These 
simulation  results  are  interesting  because  the  DPP  is  an 
effective  algorithm  as  well  as  a   proof  system.  These  simula- 
tion results  imply  that  efficient  algorithms  exist  for  these 
other  proof  systems  as  well.   Of  course  there  is  little  value 
in  running  the  DPP  and  then  simulating  the  results  in  some 
other  proof  system.  These  results  suggest  that  efficient 
algorithms  that  are  based  directly  on  these  other  proof 
systems  may  be  inferred  from  the  DPP. 

The  DPP  cannot  p-simulate  refutation  graphs.  In 
[Kowalski  and  Kuehner,  1971]  it  is  shown  that  t-linear 
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resolution  is  as  powerful  as  regular  resolution.  Corres- 
ponding to  any  t-linear  proof  there  is  a  refutation  graph 
[Shostack,  1976].   Thus,  refutation  graphs  p-simulate 
regular  resolution.   Tseitin  [Tseitin,  1968]  has  shown 
regular  tree  resolution,  and  hence,  the  DPP  cannot  p-simulate 
regular  resolution. 

From  the  point  of  view  of  power  of  proof  systems, 
regular  tree  resolution  and  the  DPP  are  identical.   The 
proofs  of  Theorems  3.2  and  3.3  demonstrate  that  a  structural 
isomorphism  exists  between  the  two  systems. 

Both  versions  of  the  Davis-Putnam  proced^lre,   DPP 
and  DPP",  are  instances  of  regular  resolution  procedures. 
The  DPP"  was  rejected  on  practical  grounds  in  favor  of  DPP. 
It  will  be  shown  that  with  respect  to  the  p-simulate 
relation   the  two  systems  are  incomparable.   (These  results 
will  be  established  for  DPP"  without  subsumption . ) 

Cook  has  shown  that  DPP"  without  subsumption  must 
generate  exponentially  long  proofs  on  the  following  class 
of  problem  instances  [Cook,  1971] .   Let  S   be  the  set  of 
purely  positive  or  purely  negative  clauses  of  length  i 
over  an  atom  set  containing  m  atoms,  p, ,...,p  .  That  is. 


^m  =  ^Pi^Pi2---PiJ^^il<i2---<i£l"^>  ^  ^Pi3^Pi2'--PiJ^-^l'^2' 


•  '   h   ^"^^ 


Note  that   S^  is  unsatisf iable  iff  m  >  2Ji-l.  If  m  <  2^-1, 
m  — 
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then  assigning  the  value  True  to  exactly  [m/2j  atoms 
assures  that  every  clause  must  contain  an  atom   made 
True  by  the  assignment  as  well  as  one  made  False.  Thus 
all  clauses  are  satisfied. 

Conversely,  if  m  >_  2£-l   then  for  any  assignment 
there  is  a  clause  left  unsatisfied.   If  half  or  more 
of  the  atoms  are  assigned  True  (False)   then  one  of  the 
purely  negative  (positive)  clauses  will  be  False. 

Cook  has  shown  that  after  the  elimination  by  the 
DPP"  (without  subsumption) ,  of  any  sequence  of  £-1  literals, 
the  number  of  clauses  in  the  resulting  set  will  grow  faster 
than  any  polynomial  in  the  size  of  the  original  set. 

Next  we  examine  the  performance  of  the  DPP  on  this 

example.   For  simplicity,  assume  that  m  is  even  and  that 

£  =  4  m.   S*^  is  then  unsatisf  iable .   The  crudest  estimate 
2       m 

of  the  length  of  the  DPP  proof  assumes  that  2  splits  are 
made  and  that  the  size  of  a  split  set  is  ^[g]r  the  size 
of  the  original  set  of  formulas.  Thus,  the  length  of  the  DPP 


proof  is  bounded  by  2 
Stirling's  formula. 


2£ 


m+1 


or  2 


2£+l 


2£ 
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Let  n  -   4  //Tr£  ;  then 
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So  the  length  of  a  DPP  proof  is  bounded  by  a  polynomial 
in  the   length  of  the  instance. 

Theorem  3.4.   DPP"  cannot  p-simulate  DPP. 

Proof .   Any  proof  of  S„   by  the  DPP" is  not  within  a  poly- 
nomial of  the  length  of  the   DPP  proof. 


D 


2  £ 

DPP"  generates  0(2   )   clauses  for  S_p  ,  most  of  which 

are  subsumed  by  others  generated.  The  DPP"  with  subsumption 

£ 
will  generate  a  short  proof  for  S„.. 


Theorem  3 . 5  (Tseitin,  Reckow) .   Regular  tree  resolution 
cannot  p-simulate  DPP". 

Corollary  3.6.   DPP  cannot  p-simulate  DPP". 

Tseitin  [Tseitin,  1968]  has  found  a  class  of  instances 

{S.}  for  which  there  exists  a  c  >  0  such  that   if  n  is  the 

1 

length  of  the  shortest  regular  resolution  proof  of  S.  , 
then  the  length  of  every  regular  tree  resolution  proof  is 
greater  than  2'^^'^°^   ^^     .      Reckow  has  remarked  that  DPP" 
without  subsumption  realizes  the  shortest  regular  resolu- 
tion  proof  of  S . .  Q 

As  these  results  show,  DPP  and  DPP"  are  substantially 
different  proof  procedures.   As  regular  resolution  procedures 
they  both  are  subject  to  Tseitin 's  result,  as  improved  by 
Galil   [Galil,   1977;  Tseitin,  1968];  namely,  there  is  a 
class  of  instances  on  which  any  regular  resolution  proof 
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en 
must  contain     2         clauses,  for  some  c  >  0,  and 

infinitely  many  n,  where  n  is  the  size  of  the  instance. 

In  Chapter  II   it  was  remarked  that  application  of 
the  splitting  rule  with  a  literal  that    occurs  frequently 
in  a  set  of  clauses  is  intuitively  desirable  because 
this  will  tend  to  minimize  the  number  of  clauses  that 
fall  into  both  split  sets.   On  the  other  hand,  such  a 
literal  will  be  a  poor  selection  of  a  literal  to  annihilate 
for  DPP"  because  a  new  clause  is  added  to  the  set   for 
every  pair  of  complementary   literals.   For   DPP" 
the  opposite  heuristic  is  applicable;  eliminate  the  literal 
that  occurs  least  frequently.   This  will  tend  to  minimize 
the  growth  of  the  set.   These  heuristic  considerations  and 
the  above  results  suggest  that  together  these  two  procedures 
will  work  very  well.   When  a  literal  that  is  a  good  candidate 
for  application  of  the  splitting  rule  is  present,  apply 
that  rule.   When  a  literal  appears  infrequently,  it  should 
be  eliminated  by  the  rule  for  elimination  of  atomic  formulas 
(DPP") . 

The  question  of  the  power  of  a  proof  system  is  important 
theoretically  because  of  the  connection  to  the  WP  =  MP 
question.   However,  since  regular  resolution  has  been  proven 
exponential,  this  question  cannot  be  answered  by  investigat- 
ing the  DPP.   The  results  of   Chapter  II  become  interesting 
in  light  of  the  weakness  of  the  DPP  as  a  proof  system.  An 
interesting  open  question  is:  Is  there  a  more  powerful  proof 
system  which  yields  an  algorithm  superior  to  the  DPP,  or  does 

the  greater  nondeterminism  of  these  systems  preclude  this 

possibility  ?  (».. 
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IV.   HORN  SETS 

4.1.  The  DPP  and  Horn  Sets 

Many  mathematical  statements  are  of  the  form 

(A^  &At&...&A)->B,  where  At,A„,...,A  ,  B  are 
1     2  n  12n 

atoms,  i.e.  given  hypotheses   A, ,A„,.,.,A    the  conclu- 
sion B  follows.   When  converted  to  CNF,  statements  of 
this  form  become  a  single  conjunct.  A,  A2 . . .  A   B  . 
A  clause  with  at  most  one  positive  literal  is  a  Horn   clause. 
A  set  containing  exclusively  Horn  clauses  is  a  Horn    set. 
A  theorem  prover  dealing  with  ordinary  mathematics  will 
encounter  sets  made  up  predominantly  of  Horn  clauses,  and, 
in  fact,  the  sets  encountered  will  often  be  Horn  sets. 
As  a  result,  Horn  sets  have  undergone  special  study. 
A  unit    (input)    resolution    derivation    is  a  resolution  deriva- 
tion in  which  one  parent  clause  of  every  resolvent  is  a  unit 
(input)  clause.   In  [Henschen  and  Wos ,  1974]  it  is  shown 
that  if  S  is  an  unsatisf iable  Horn  set,  then  S  has  both  unit 
and  input  resolution  refutations.   It  follows  that  any  unsat- 
isf iable  set  must  contain  a  unit  clause.   Thus  the  DPP, 
given  an  unsatisf iable   Horn  set  will  on  the  first  step 
be  able  to  apply  the  unit  rule.   The  set  S,  derived  from 
application  of  this  rule  will  also  be  an  unsatisf iable 
Horn  set,  since  every  clause  in  S,  is  a  subset  of  a  clause 
in  S.   S,  must  also  contain  a  unit  clause.   It  follows  that 
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at  every  step  of  the  DPP,  the  unit  rule  will  be  applicable. 
If  at  any  step  the  unit  rule  does  not  apply  to  a  Horn  set, 
the  set  must  be  satisfiable.   Construction  of  an  efficient 
algorithm  to  test  the  satisfiability  of  Horn  sets  can  be 
based  on  the  DPP  utilizing  these  observations.   The  algorithm 
simply  continues   to  apply  the  unit  rule  until  the  set  is 
shown  to  be  unsatisf iable  or  the  unit  rule  cannot  be  applied. 
There  is  no  necessity  to  consider  other  rules.   The  choice 
of  data  structures  to  represent  S  can  take  advantage  of  this 
situation.   Clauses  are  represented  as  pointer  structures. 
Each  clause  contains  a  field  holding  its  current  length. 
All  the  literals  occurring  in  S  associated  with  the  same 
atom  are  linked  together.   The  presence  of  a  literal  in  a 
clause  is  indicated  by  a  pointer  to  the  next  clause  contain- 
ing that  literal  (or  its  complement) .   Using  this  data 
structure,  application  of  the  unit  rule  is  very  efficient. 
Starting  at  the  unit  clause,  the  links  from  the  unit  literal 
are  followed  to  its  occurrences  in  the  other  clauses  either 
marking  the  literal  deleted  from  these  clauses  and  decreasing 
the  length  of  the  clause  by  one,  or  dropping  the  clause 
entirely  (by  making  its  length  zero) .   As  the  links  are 
followed   all  new  unit  clauses  are  noted  so  they  can  be  used 
to  initiate   subsequent  steps.   With  this  representation 
every  literal  is  examined  at  most  once.   This  establishes 
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Theorem  4.1.  Horn  sets  can  be  tested  for  satisfiability 
in  linear  time. 

It  will  nevertheless  be  efficient  to  apply  the  DPP 
without  any  special  modifications  for  Horn  sets,  but  the  best 
upper  bound  obtainable  for  the  worst  case  performance  will 
be  cubic  in  the  size  of  the  set.   The  complexity  goes  up 
one  degree  because  of  the  lack  of  any  special  data  structur- 
ing which  allows  for  unit  rule  application  in   sublinear 
time.   Another   degree  will  be  lost  because  a  set  will  not 
be  found  satisfiable   if  the  unit  rule  does  not  apply.  The 
splitting  rule  will  be  applied  and  the  algorithm  will  proceed 
normally,  finding  a  satisfying  assignment. 

2 
Lemma  4.2.   The  DPP  will  run  in  0(£  )  time  on  unsatisf lable 

Horn  sets  of  size  £. 

Proof.   At  each  step  the  unit  rule  will  be  applied. 

This  takes  time  linear  in  the  length  of  the  set.   Since 

the  unit  rule  leaves  only  one  set  to  test,  the  rule  will  be 

applied  r  times,  where  r  is  the  number  of  distinct  atoms 

2 
in  the  set.   Since  r  ;^  £   the  algorithm  runs  in  0{l    )  time. 

Theorem  4.3.   The  DPP,  without  modification,  will  take  at 
most  cubic  time,  in  the  worst  case,  to  test  the  satisfi- 
ability of  Horn  sets. 

Proof.  The  DPP  performs  a  depth  first  search  of  possible 
satisfying  valuations.  If  it  ever  finds  such  a  valuation 
it  halts.   The  DPP  realizes  the  worst  case  for  satisfiable 
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Horn  sets   when  only  the  last  examined  valuation  satisfies 
the  set.   For  this  to  occur,  whenever  the  splitting  rule 
is  applied   the  first  set  tested  (say  S, )  must  be  unsatis- 
fiable   and  the  second  S„  must  be  satisfiable.   Let  i      be 
the  length  of  a  given  set  S.   Let  G{1)    be  the  worst  case 
time  complexity  of  the  DPP  on  Horn  sets.   S,  is  unsatis- 

fiable,  Horn  and  has  a  maximum  length  of  £,-3.   Therefore, 

2 

by  Lemma  4.2,  S   is  found  unsatisf iable  in  time  k{)l-3) 

for  some  constant  k.  Thus  we  get  the  following  recurrence 
for  G(£)  : 

(ci    +  k(£  -  3)^  +  G(£  -  1)  ,  I    >    2 
G(£)  =  \ 

0  otherwise 


Aj  X"  '™  J  *-\  -^ 

G(£)  <_  ^   ci  +  I      k  i   ,  which  is  0(£  ) 
1=1       i=l 


n 


whether  this  upper  bound  is  achieved  remains  an  open 
problem.    It  is  reasonable  to  expect  very  good  (quadratic) 
performance  by  the  DPP  on  Horn  sets. 
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4.2.  Proof  Systems   for  Horn  Sets 

It  is  well  known  [Chang  and  Lee,  1973]  that  a  set    of 
clauses  has  a  unit  refutation  iff  it  has  an  input  refuta- 
tion.  Horn  sets  have  unit  and  input  refutations.  A  converse 
can  be  stated  with  the  aid  of  a  few  definitions.  Given  a 
set  S  of  clauses  and  a  set  A  of  atoms  occurring  in  S ,  a 
renaming    of  S  is  the  set  of  clauses  resulting  from  the 
replacement  in  S  of  every  literal  whose  atom  appears  in  A 
by  its  negation.   Renaming  preserves  satisfiability.  A 
set  of  clauses  is  minimally    unsatisfiable    if  it  is 
unsatisf iable,  but  any  proper  subset  of  it  is  not.  If  S  is 
minimally   unsatisfiable  and  has  a  unit  refutation  then  S 
is  the  renaming  of  a  Horn  set  [Henshen  and  Wos,  1974]. 
The  minimality  condition  is  natural  since  an  unsatisfiable 
Horn  set  can  be  trivially  augmented  by  a  non-Horn  clause 
and  still  have  a  unit  refutation.   We  can   conclude  that 
S  is  a  Horn  set  if  S  has  a  unit  refutation   where  the 
unit  clauses  are  positive  literals.   From  the  viewpoint  of 
studying  proof  systems,  an  investigation  of  unit  and  input 
refutations   can  be  made  by  examining  their  behavior 
exclusively  on  Horn  sets.   As  proof  systems,  input  and 
unit  resolution   are  incomplete.   Although  they  are  both 
powerful  systems  over  the  domain  for  which  they  are  complete 
(i.e.  sets  renamable  as    Horn  sets),  we  shall  see  that  input 
resolution    proofs    are  generally  shorter  than  unit  proofs. 


72 


Input  resolution  corresponds  closely  to  the  generation  of 
resolution  refutations  using  the  DPP;  unit  resolution 
corresponds  to  the  use  of  the  DPP". 

Theorem  4.4.   Let  S  be  an  unsatisf iable  Horn  set  of  length 
i    (£  is  the  total  number  of  literals  in  S) .   S  has  a  unit 
resolution  refutation  in  which  there  are  at  most  i   applica- 
tions of  the  resolution  rule. 

Proof.   Consider  the  DPP"  refutation  of  S  in  which  at  each 
step  the  atom  corresponding  to  the  unit  clause  is  annihilated, 
Assuming  that  all  subsumed  clauses  have  been  removed,  the 
unit  clause  containing  the  literal  L  will  be  the  only 
occurrence  of  L  in  the  set.   Thus,  a  step  of  DPP"  will 
resolve  L  with  every  clause  containing  L  and  then  drop  all 
the  parents  of  the  resolutions.   If  k  of  the  clauses  contain 
L,  k  resolutions  will  be  performed;  the  resulting  set,  S,  , 
will  have  length  2,-k-l  (k  literals  and  the  unit  clause  are 
removed) .   The  same  procedure  is  repeated  at  the  next  step. 
It  is  easily  seen  that  at  most  i    resolutions  are  possible. 
More  precisely,  if  r  is  the  number  of  distinct  atoms  in  S, 
and  hence  the  number  of  steps   performed,  then  the  maximum 
number  of  resolutions  is  £-r.  q 

The  sharper  bound,  £  -  r,  is    attained  by  the  set 

S  =  {p-^rP-J^2'     •"'    Pr-l^r  '  ^r^  ' 
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Theorem  4.5.  Let  S  be  an  unsatisf iable  Horn  set  with  r 
distinct  literals.   Then  there  is  an  input  resolution 
refutation   with  at  most  r  applications  of  the 
resolution  rule. 

Proof.   Consider  the  resolution  refutation  obtainable 
from  the  DPP  procedure.   We  have  previously  observed  that 
for  an  unsatisf iable  Horn  set,  DPP  will  apply  the  unit 
rule  at  every  step.   There  will  be  one  resolution  for  each 
application  of  a  rule  by  the  DPP.   Since  the    unit  rule 
is  always  applicable  it  will  be  applied  at  most  r  times. 
The  resolution  corresponding  to  application   of  the  unit 
rule  always  has  an  input  clause  as  one  of   its  parents 
(see  Figure  3.1),   Thus  the  resolution  refutation  produced 
will  be  an  input  refutation  involving  at  most  r  resolu- 
tions. 

This  upper  bound  represents  the  best   performance  that 
could  be  expected  for  a  resolution  proof.   If  a  set  is 
minimally  unsatisf iable ,  every  clause  must  be  a  parent  for 
a  resolution  and  so  every  atom  occurring  must  be  annihilated 
This  means  that  r  resolutions  is  minimal. 
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4 . 3   Renaming 

Recall  that  a  renaming  of  a  set  of  clauses  is  a  set 
obtained   by  negating  all  the  occurrences  of  literals 
belonging  to  selected  atoms  [Meltzer,  1966].   A  set  is 
renamabte   as    a   Horn    set    if  it  has  a  renaming  which  is  a  Horn 
set.   Sets  that  are  renamable  as  Horn  sets  can  be  tested 
for  satisfiability  by  the  DPP  as  efficiently  as  Horn  sets. 
It  is  trivial  to  determine  if  a  set  is  Horn  but  not  as  easy 
to  determine  if  a  set  is  renamable  as  a  Horn  set.  There  are 
2^   possible  renamings ;  if  one  had  to  enumerate  each  one, 
one  could  just  as  well  enumerate  all  truth  assignments 
and  test  satisfiability.   Fortunately,  testing  whether  a 
set  is  renamable  as  a  Horn  set  can  be  reduced  to  testing 

2-satisf lability  (testing  satisfiability  of  sets  with  at 

* 
most  two  literals  per  clause) .    2-satisf lability  is  decidable 

in  polynomial  time.   The  problem  of  renamability  as  a 

Horn  set  is  equivalent  to  the  problem  of  "all  but  one 

satisfiability,"  that  is,  given  a  set  of  clauses,  is  there 

a  valuation  which  makes  each  literal,  except  possibly  one 

per  clause,  true?   The  equivalence  is  seen  by  assigning  a 

literal  the   value  True  in  the  valuation  iff  the  literal 

is  negated  by  the  renaming.   The  problem  of  "all  but  one 

satisfiability"  is  easily  seen  to  be  reducible  to  2-satisfi- 

ability  by  forming  all  clauses  of  pairs  of  literals  that 


This  result  was  obtained  independently  by  Lewis  [Lewis,  1978] 
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appear  in  the  same  clause  of  the  given  set.  The  set  derived 
in  this  fashion  is  satisfiable  iff  the  given  set  is  "all  but 
one  satisfiable".   A  formal   proof  follows. 

Theorem  4.6.   In  polynomial  time  it  can  be  determined  if 
a  set  is  renamable  as  a  Horn  set. 

Proof.   Given  S  =  [C    ,C^, . . . ,C^} ,    with 


C.  =  {L.^,L.2,L.3 


/  • 


'^i,n.^'   Wj  ^  {x^,...,x^,x^ x^}. 


define  a  new  set  Q   ,  such  that  Q   is  satisfiable  iff  S 

s  s 

is  renamable  as  a  Horn  set.   Q  will  consist  of  the  follow- 

s 

ing  clauses: 

^ij  ^ik  '    llj<k^n^,    i=l,...,n. 

Suppose  Q   is  satisfiable.  Define  a  renaming  of  S  in  which 

the  atoms  made  True  by  the  valuation  are  those  whose  literals 

are  renamed.   Suppose  that  a  clause  C.  is  not  Horn  after 

renaming,  and  that  the  two  positive  literals  in  C.  derive 

from  L. .L.,  .   If,  say,  L. .  ,  was  negative  it  must  have  been 
1]  ik         -^  '   13  '        ^ 

renamed   and  so  the  atom  associated  with  it  is  given  the 
value  True.   Hence  L. .  is  given  the  value  False.   If  L. . 
was  positive,  it  was  not  renamed  and  is  therefore  given 
the  value  False  by  the  valuation.   The  same  argument  applies 
to  L  •  1^  •   •'-'i-i-'-'ik  ^^  then  made  False  by  the  valuation,  a  contra- 
diction.  Conversely,  given  a  renaming  of  S,  define  a 
valuation  for  Q   by  assigning  the  value  True  to  atoms  whose 
literals  are  negated  and  the  value  False  to  all  others. 
If  a  clause  L. .L.   in  Q   is  False  in  this  valuation,  then 

X  J  IK        S 
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clause  C.  in  S  will  not  be  Horn.   Thus  Q   is  satisfiable 
1  s 

iff  S  is  renamable  as  a  Horn  set.   Since  Q   contains  at 

2 
most  nr   clauses  and  because  satisfiability  of  Q   is 

^  s 

decidable  in  quadratic  time,  decidability  of  the  renaming 
of  S  as  a  Horn  set  is  polynomial.  p. 

We  have  demonstrated  efficient  algorithms  for  testing 
the  satisfiability  of  Horn  sets  as  well  as  sets  renamable 
as  Horn  sets.   Most  sets  occurring  in  mathematical  contexts 
will  consist  predominantly  of  Horn  clauses  but  will  not 
be  Horn  or  renamable  as  a  Horn  set.   To  test  the  satisfi- 
ability of  such  a  set  by  the  DPP,  a  natural  strategy  is 
to  apply  the  splitting  rule  to  the  excess  positive   literals 

(those  positive  literals  beyond  the  one  allowed  in  a  Horn 
clause)  of  the  set  (this  approach  is  also  suggested  in 

[Schwartz,  1978]).   After  these  splits  are  made,  the  DPP 

will  only  need  to  be, applied  to  Horn  sets.   This  strategy 

corresponds  to  a  proof  by  cases  in  a  mathematical  argument 

in  which  one  of   the  hypotheses  (or  an  invoked  theorem)  is 

of  the  form  (A   &  A   &  ...  &  A  )  ^  (B,  V  B2).   When  put 

in  CNF  this  statement  becomes  the  clause  A,  A^  ...  A   B,B„  . 

I      Z  n   i  z 

Splitting  on  the  excess  literal,  B„  ,  corresponds  to  separately 
considering  the  two  cases  (  A^  &  ...  &  A  )  ->  B,  and 
(A,  &  ...  &  A  )  -*   ^2    •    '^l^is  procedure  will  be  exponential 
only  in  the  number  of  distinct  excess  positive  literals 
that  appear  in  a  set.   This  leads  one  to  expect  good  perform- 
ance in  most  situations. 
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Renaming  may  be  applied  to  reduce  the  number  of 
distinct  excess  positive  literals.   Specifically,  given  a 
set  S,  can  we  efficiently  find  the  minimum   k  and  a  set  E, 
with  |e|  =  k,  so  that  removal  of  the  atoms  in  E  from  S 
result   in  a  set  renamable  as  a  Horn  set?   This  is  an  open 
problem,  but  evidence  indicates  that  the  answer  is  negative. 
Indeed  related  problems  that  are  apparently  easier  can  be 
shown  to  be  NP-hard. 

Theorem  4.7.    The  problem  of  finding  a  renaming  which 
minimizes  the  number  of   non-Horn  clauses  is  NP-hard. 

Proof.   We  reduce  maximum  2-satisf lability  [Garey,  Johnson 
and  Stockmeyer,  1976]  to  this  minimization  problem.  The 
reduction  is  trivial.   Renaming  is  equivalent  to  "all  but 
one  satisfiability",  which,  in  the  case  of  clauses  contain- 
ing at  most  2  literals,  is  identical  to  satisfiability. 
Thus,  maximizing  the  number  of  clauses  that  are  satisfied 
by  a  valuation  is  identical  to  maximizing  the  number  of 
Horn  clauses  in  a  renaming.  g 

Corollary  4.8.  Minimizing   the  number  of  excess  (not  neces- 
sarily distinct)  positive  literals  by  renaming  is  NP-hard. 

Proof.   In  the  special  case  of  2  literal   clauses  this  is 
identical  to  minimizing  the  number  of  non-Horn  clauses.    ^ 
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